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CHAPTER  1 
PROBLEM  STATEMENT 


It  is  0450  AM  and  the  sky  is  just  beginning  to  show  the  dawn 
of  a  new  day.  Sp4  Eagle-eye  and  PVT  Sharpshooter  have  been 
awake  for  the  last  50  minutes  scanning  the  sky  for  hostile  aircraft. 

They  are  one  of  the  Stinger  Teams  supporting  1-62  Infantry 
Battalion  (Mechanized).  The  battalion  is  in  an  assembly  area 
preparing  for  a  movement  to  contact  at  0500  hours.  The  battalion  is 
considered  a  high  priority  target  for  enemy  aircraft  (due  to  their 
vulnerability  at  this  phase  of  the  operation).  An  intelligence  report 
indicating  that  the  enemy  will  attack  this  morning  has  raised  the  hair 
on  the  back  of  Sharpshooter’s  neck.  Eagle-eye  and  Sharpshooter 
know  the  enemy  likes  to  attack  assembly  areas  and  do  so  early  in 
the  morning  to  take  advantage  of  both  the  congestion  of  assembly 
areas  and  degraded  visibility  of  dawn.  They  are  ready! 

The  job  of  this  air  defense  team  is  to  acquire  and  quickly 
engage  enemy  aircraft  before  severe  damage  can  be  caused  to  the 
battalion.  Suddenly,  as  if  on  cue,  a  voice  over  the  radio  squalks: 
"Dynamite  from  the  east.  Dynamite  from  the  east!  (code  word  for 
hostile  aircraft).  PVT  Sharpshooter,  the  gunner,  moves  quickly  into 
his  hasty  firing  position  on  top  of  their  armored  personnel  carrier. 

His  job  is  to  acquire  the  target,  track  it  until  it  is  within  range,  and 
engage  upon  command  from  his  team  chief.  Sp4  Eagle-eye  is  the 
team  chief.  His  job  is  to  acquire  the  target,  make  a  target 
identification  and  give  his  gunner  the  command  to  engage  (shoot)  or 
not. 
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From  a  cognitive  perspective.  Eagle-eye  must  accomplish  four 
tasks:  (1)  acquire  the  aircraft,  (2)  identify  the  aircraft  (the  target 
aircraft  is  classified  as  friendly,  hostile  or  unknown),  (3)  make  an 
engagement  decision  (i.e.  to  fire  or  not)  and  (4)  respond  accordingly 
(give  the  appropriate  command  to  his  gunner).  These  cognitive 
components  can  be  analyzed  within  the  framework  of  the 
information  processing  approach  and  more  specifically  a  schema 
model.  Schema  theory  involves  matching  the  perceived  material 
with  a  structured  mental  representation  of  the  user's  general 
knowledge  of  the  topic.  A  schema  provides  a  basis  for  categorizing, 
selecting,  deleting,  abstracting,  consolidating  and  organizing 
.information.  Once  learned,  it  can  be  employed  to  comprehend  other 
situations  with  similar  structures.  A  schema  then  is  an  ordered 
checklist  or  analytical  template  which  structures  perception  and 
information  processing  (Hopple  and  Halpin,  19S5),  represents  the 
combination  of  cognitive  components  required  for  a  task  as  well  as 
the  order  in  which  they  are  to  be  accomplished. 

The  schema  for  this  task  would  be  as  follows: 

(1)  Target  Acquisition.  A  need  for  action  is  generated  when 
the  aircraft  is  spotted.  This  is  the  detection  of  the  presence  of  an 
aircraft. 

(2)  Target  Identification/Recognition.  This  again  involves  the 
perceptual  processes  in  a  data-driven  bottom-up  fashion.  A  feature 
matching  process  occurs  between  the  target  aircraft  and  a  memory 
set  of  previously  learned  aircraft.  The  search  can  result  in  a  match 
(identifying  the  target  as  friendly  or  hostile)  or  no  match  (unknown). 
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(3)  Decision  Making.  This  component  determines  a  course  of 
action.  An  action  is  determined  in  a  rule-based  fashion  where  pre- 
established  rules  are  applied.  The  rules  combine  aircraft  type  and 
current  weapon  control  mode  to  arrive  at  the  appropriate  decision. 

(4)  Response.  The  decision  is  carryed  out  through  some 
physical  action  (i.e.  giving  the  order  to  fire  or  not). 

In  this  task.  Eagle-eye  is  required  to  visually  identify  the 
aircraft  before  giving  the  command  to  fire.  Even  with  this  doctrinal 
constraint,  Sp4  Eagle-eye  would  welcome  some  assistance. 

At  one  level,  he  does  not  require  assistance  to  accomplish  this 
task.  He  can  operationally  perform  all  of  the  components  of  the  task 
and  determine  an  ultimate  decision  individually,  which  is  the  current 
method  of  accomplishing  this  task.  But  on  another  level,  considering 
the  speed  of  current  aircraft,  their  increased  weapon  ranges  and  the 
possibility  of  degraded  visibility  (from  weather  or  smoke),  some  type 
of  assistance  (i.e.  information  on  the  aircraft)  might  enhance  his 
ability  to  make  a  faster  and  more  accurate  decision. 

There  appears  a  need  for  some  sort  of  decision  support  system 
(decision  aid)  to  assist  Eagle-eye  in  this  decision  task.  The  nature  of 
this  decision  environment  (i.e.  short  time  frame  to  make  the  decision 
and  high  cost  of  an  error)  makes  this  assistance  all  the  more  crucial. 

In  general,  during  the  last  ten  to  twenty  years,  command, 
control,  communication,  and  intelligence  (C3I)  has  become 
increasingly  important  to  the  war  fighting  capability  of  the  military 
forces  (Tachmindji,  1986).  Military  decision  makers  will  be 
inundated  with  voluminous  amounts  of  information  which  will  have 
to  be  processed  rapidly  to  enable  leaders  to  act  within  the  decision 
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cycle  of  the  enemy.  Expert  systems  and  other  artificial  intelligence 
applications  will  become  a  necessity  (Wolfe,  1986).  Each  of  the 
services  has  spent  millions  of  dollars  over  the  last  decade  attempting 
to  acquire  tactical  command  and  control  systems  that  would  provide 
centralized  information  for  the  decisions  maker's  use  in  planning  and 
control  of  forces  (Teates,  1986).  The  Defense  Advanced  Research 
Project  Agency  (DARPA)  and  the  Strategic  Defense  Initiative 
Organization  (SDIO)  are  supporting  work  in  these  areas  (Shumaker, 
1987).  In  particular,  the  army  is  currently  working  on  a  hand  held 
device  to  assist  the  Stinger  team  chief  in  the  field.  The  tradeoffs 
associated  with  this  sort  of  decision  support  system  are  the  quicker 
and  more  accurate  engagement  of  aircraft  versus  the  cost  of  such  an 
aid. 

This  research  is  motivated  by  an  interest  in  human  interaction 
with  decision  support  systems  and  the  more  effective 
accomplishment  of  the  Air  Defender's  mission.  It  has  led  to  the 
investigation  of  one  of  the  aspects  of  such  a  decision  aid  (the 
cognitive  content  of  the  information  conveyed  by  the  decision  aid) 
and  human  interaction  with  decision  aids  in  general.  The  objectives 
of  this  research  are  to  determine: 

(1)  the  effect  of  decision  aiding  on  decision  making 
performance  in  a  military  command  and  control  situation  and 

(2)  the  cognitive  content  of  the  information  that  is  most 
effective  in  this  situation. 
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CHAPTER  2 
LITERATURE  REVIEW 


There  are  a  number  of  areas  in  the  literature  related  to  the 
effect  of  the  type  of  information  presented  to  a  user  by  a  decision 
support  system  on  the  total  system  performance.  In  this  section 
those  findings  and  how  they  relate  to  this  research  will  be  discussed. 
Those  areas  are:  human-computer  interaction  in  complex  dynamic 
systems;  decision  support  technology,  with  emphasis  on  the  human 
interaction  with  the  system;  and  the  cognitive  psychology  research 
on  semantic  priming. 

) 


There  is  an  increasing  trend  towards  a  substantial  use  of 
automation  in  the  control  of  complex,  dynamic  systems.  The  result  is 
larger,  more  complex  systems  with  centralized  control  *{W<x>ds  -and 
^Rothr  1987;'  Rasmussen,  19$6).  This  increasing  complexity  of 
technological  systems  and  increased  use  of  automation  has  shifted 
the  demands  on  human  performance  from  a  mechanical/physical 

level  to  a  cognitive  level.  - — 

A  key  scheme  for  classifying  levels  of  technology  described  by 
Kantowicz  and  Sorkin  (1983,  p.  7-9)  focuses  on  the  contributions  of 
people  and  machines  to  the  total  system.  At  the  lowest  level  the 
person  supplies  both  the  power  and  control  of  the  system.  A  human 
using  a  shovel  is  functioning  at  this  level.  At  the  next  level,  the 
machine  provides  the  power  and  the  human  provides  the  control. 
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An  example  of  this  is  an  operator  using  punch  press.  third  level 

has  the  machine  providing  power  and  information,  while  the  human 
controls  the  functions.  An  operation  in  a  process  control  plant  is  a 
good  example  of  this.  Finally,  the  highest  level  of  technology  has  the 
machine  supply  the  power,  information,  and  control,  while  the 
human  monitors  the  operation.  A  pilot  flying  an  airplane  on 
autopilot  is  an  example  of  this  level.  This  scheme  clearly  illustrates 
the  changes  in  the  human's  contribution  with  the  increased  use  of 
automation.  The  human  has  evolved  from  a  doer  into  an  information 
processor.  The  roles  of  automation  and  the  human  in  these  complex, 
dynamic  systems  will  now  be  discussed. 

Role  of  Automation 

The  role  of  automation  in  these  complex,  dynamic  systems  is  to 
provide  more  powerful  and  efficient  control.  Automation  takes  over 
routine  tasks  such  as  data  sensing,  aggregation,  and  manipulation,  as 
well  as  simple  decision  making,  given  well  defined  alternatives 
(Mitchell,  1986).  The  introduction  of  automation  to  accomplish  these 
functions  has  changed  the  role  of  the  human. 

Role . of  the  Human 

The  new  role  of  the  human  is  to  act  as  a  supervisor  of  the 
system  with  goal  setting,  problem  solving  and  decision  making  as  the 
primary  tasks.  The  human's  role  is  primarily  passive,  involving  the 
monitoring  of  the  system  state.  This  role  only  changes  when 
unexpected  system  events  or  failures  occur  (Mitchell,  1987).  The 
human  supervisor  .will  carry  out  the  functions  that  previously 
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required  several  people.  The  major  function  of  the  human  operator 
is  information  processing.  The  human  operator  transforms  displayed 
data  into  meaningful  information  and  eventually  into  control  actions 
(Mitchell,  1983).  A  key  issue  that  arises  with  this  new  human  role 
and  function  is  the  way  that  information  is  displayed  to  the  human 
(Mitchell,  1986).  This  issue  appears  to  be  of  primary  importance  to 
the  cognitive  interaction  between  human  and  the  automation 
(computer)  in  these  complex,  dynamic  system  environments.  To 
understand  this  interaction  more  fully,  it  is  important  to  discuss  the 
strengths  and  limitations  of  both  automation  and  the  human. 

Strengths  and  Limitations  of  Automation 

The  strengths  of  automation  will  be  addressed  first,  followed 
by  the  limitations.  In  this  section,  the  terms  computer  and 
automation  will  be  used  interchangeably.  In  general,  computers 
(1)  do  not  become  complacent  or  yield  to  distraction,  (2)  remember 
without  error  (save  mechanical  malfunction),  (3)  do  not  fatigue  and 
can  monitor  continuously,  (4)  are  not  susceptible  to  context  effects, 
and  (5)  they  are  usually  not  disturbed  or  degraded  by  environmental 
conditions  (Chambers  and  Nagel,  1985).  Also,  computers  can  retrieve 
coded  information  quickly  and  accurately  when  specifically 
requested  (McCormick,  1976,  p.  261-262). 

Computers  do  have  some  limitations  however.  They  often  do 
not  indicate  when  something  has  gone  amiss  and  that  the  limits  of 
their  capabilities  have  been  exceeded  (Borning,  1987).  More 
importantly,  computers  are  unable  to  effectively  handle  degraded 
information  or  novel  situations  with  ambiguous  conditions  in  which 
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previous  guidance  of  an  explicit  nature  is  not  available.  In  other 
words,  their  precision  and  reliability  is  impressive  but  in  no  way 
challenges  the  human  intellect.  They  respond  well  to  those  situations 
anticipated  by  the  system  designers  (Chambers  and  Nagel,  1985). 

Strengths  and  Limitations  of  the  Human 

The  strengths  of  humans  lie  mainly  in  their  ability  to  be 
creative  and  reason  inductively.  Humans  have  a  good  ability  to  cope 
with  ambiguous,  vague  or  uncertain  environments.  They  make 
inductive  decisions  in  novel  situations  and  generalize  from  analogous 
experiences,  allowing  them  to  improvise  and  exercise  judgement 
(Chambers  and  Nagel,  1985).  Humans  are  good  at  remembering 
principles  and  strategies,  and  then  drawing  upon  these  to  make 
decisions  and  generate  solutions  to  varied  problems.  In  addition, 
they  are  also  good  at  developing  entirely  new  solutions  (McCormick, 
1976,  p.  261-262). 

There  are  a  number  of  limitations  that  have  been  observed  in 
humans  in  complex  system  environments  that  are  important  to 
discuss  because  these  limitations  are  instrumental  in  the  design  of 
the  human-computer  interface.  Those  limitations  focus  on  the 
information  processing  ability  of  humans.  First,  human  ability  to 
monitor  changing  information  is  fairly  low.  Monty  (1973)  found  that 
subjects  would  make  errors  even  in  situations  where  they  had  to 
simultaneously  monitor  only  two  or  three  items  of  information. 
Second,  studies  of  aircraft  and  helicopter  pilots  indicated  an  extreme 
difficulty  in  integrating  large  quantities  of  displayed  data  (Murphy, 
McGee,  Palmer,  Paulk  and  Wempe,  1978).  In  a  similar  type  of 


8 


experiment,  Bejczy  and  Paine  (1977)  showed  that  the  additional 
effort  expended  in  monitoring  and  integrating  a  larger  input 
(number  of  sensors)  is  not  offset  by  the  increase  in  the  completeness 
of  information.  The  greater  demands  on  the  human  information 
processing  system  resulted  in  degraded  performance.  Third, 
research  by  Rouse  (1978a,  1978b,  1979)  suggests  that  humans  have 
a  difficult  time  synthesizing  meaningful  information  at  the 
conceptual  level  from  individual  data  items,  and  Laitos  (1978) 
showed  that  such  performance  further  deteriorates  under  stressful 
circumstances.  Blumenthal  (1977)  also  found  that  humans  are 
limited  by  the  capacity  of  short  term  memory  and  the  difficulty  in 
processing,  synthesizing,  and  integrating  multiple  data  items,  as  well 
as  time  constrains.  Lastly,  the  research  by  Tversky  and  Kahneman 
(1981)  showed  that  human  decision  making  is  affected  by  the  way  in 
which  the  situation  is  presented.  Different  decisions  will  be  made 
based  on  the  same  information  depending  on  whether  it  is  presented 
in  a  positive  or  negative  manner. 

Mitchell  (1983,  p.  353)  summarizes  the  implications  of  these 
limitations  in  this  way.  "The  underlying  message  is  clear:  the  human 
operator  in  complex  systems  can  be  expected  to  act  with  competence 
and  confidence  only  when  needed  information  is  provided  in  a 
timely  manner  in  a  readably  interpretable  form."  She  recommends 
the  use  of  integrated  computer-based  information  displays  to 
provide  operators  with  information  that  is  more  compatible  with 
their  higher  level  information  needs. 
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Summary 

Based  on  the  strengths  of  the  human  and  limitations  of 
automation,  the  human  operator  is  likely  to  remain  a  critical 
component  in  the  control  of  complex,  dynamic  systems.  Therefore,  a 
major  design  issue  is  how  to  use  automation  to  enhance  the 
effectiveness  of  the  human  operator.  The  goal  is  not  to  replace  the 
human  operator  but  rather  to  use  automation  to  amplify  human 
skills  (Mitchell,  1987;  Woods  and  Roth,  1987).  Chambers  and  Nagel 
(198S)  identify  some  major  research  and  design  issues  that  must  be 
addressed  in  the  use  of  advanced  automation.  Those  issues  are:  (1) 
locus  of  control,  (2)  manual  reversion,  (3)  error  tolerance,  (4) 
information  transfer,  (5)  allocation  of  functions  between  the  human 
and  automation,  (6)  training,  (7)  operator/crew  acceptance,  and  (8) 
decision  aiding. 

Of  the  new  roles  acquired  by  the  human,  decision  making  has 
emerged  as  the  focal  point  of  the  often  conflicting  demands  on 
human  action.  Chambers  and  Nagel  have  identified  one  of  the 
functions  automation  can  perform  in  complex,  joint  human- 
automation  systems:  aiding  the  human  in  the  task  of  decision 
making.  To  enhance  decision  making  performance,  one  might 
incorporate  some  type  of  automated  decision  support  system. 
Advances  in  decision  support  systems  (e.g.  expert  systems)  have  now 
provided  that  possibility  in  the  quest  to  support  human  performance. 
Woods  (1986)  explains  that  these  intelligent  decision  support 
systems  strive  to  exploit  the  strengths  of  both  the  human  and  the 
computer  components  of  complex  systems.  The  literature  on  these 
decision  support  systems  will  now  be  addressed. 
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There  are  a  number  of  important  issues  related  to  the  use  of 
decision  support  technology  in  process  control  and  military  command 
and  control,  and  the  role  of  the  human  in  these  systems.  Some  of 
these  issues  will  now  be  addressed. 
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Given  that  the  strengths  and  limitations  of  both  the  human  and 
automation  are  known,  and  adhering  to  the  assumption  that  the  use 
of  automation  to  amplify  human  skills  is  desirable;  the  first 
determination  to  be  made  is  whether  decision  aiding  is  warranted  for 
the  specific  system/situation  under  consideration.  Hopple  (1986) 
discusses  the'  generalized  assumption  that  computerization  is  always 
beneficial.  He  called  this  "methidolatry."  He  suggests  a  framework 
for  determining  the  aidability  of  the  situation  and  the  type  of  aiding 
that  would  most  enhance  total  system  performance.  He  considers 
two  variable  clusters:  (1)  those  variables  associated  with  the  decision 
situation  and  (2)  those  associated  with  decision  making  functions  or 
tasks.  The  resulting  matrix  constitutes  the  decision  making 
opportunities.  This  framework  will  now  be  elaborated  on. 

The  decision  situation  variables  focus  on  the  complexity  of  the 
decision  to  be  made  and  the  quality  of  the  data  input.  He  defines 
two  general  levels  of  decision  complexity:  (I)  closed  decisions — where 
decisions  are  both  frequent  and  familiar  and  the  course  of  action  is 
straight  forward  (i.e.,  observed  patterns  of  information  are  mapped 
directly  to  appropriate  courses  of  action);  and  (2)  open  decisions — 


where  no  self  evident,  obvious,  a  priori  bounds  exist  on  the  possible 
solution  set.  The  data  input  quality  is  given  values  of  high  or  low. 

Hopple  clearly  states  that  decision  aiding  systems  are 
potentially  feasible  in  all  of  the  combinations,  however  the  focus  of 
the  decision  aid  changes.  He  does,  however,  believe  that  decision 
aiding  is  most  needed  when  input  data  quality  is  low  and  decision 
making  time  is  limited  (time  stress). 

The  second  variable  is  the  set  of  decision  making  tasks  or 
functions.  The  following  breakdown  of  the  four  major  decision 
making  tasks  is  taken  from  Wohl  (1981): 

(1)  stimulus/data  exposure  or  gathering 

(2)  hypotheses/alternatives  generation 

(3)  analysis  and  selection  of  one  of  the  alternatives 

(4)  response 

Hopple  uses  this  two  variable  typology  to  suggest  decision 
aiding  application  domains  corresponding  to  steps  in  the  decision 
making  sequence.  This  typology  allows  the  determination  of  the 
need  for  decision  aiding,  the  aiding  functions  desirable  (assimilation, 
interpretation,  analysis,  situation  assessment,  alternative  generation, 
or  solution  recommendation),  and  which  decision  making  tasks  to  aid. 

Hopple's  framework  is  generic  in  nature,  but  can  be  used  to 
determine  aidability  in  the  arena  of  military  command  and  control. 

In  the  military,  command  and  control  is  perhaps  the  ultimate  force 
multiplier  (Heilmeier,  1986).  The  essence  of  command  and  control  is 
decision  making,  and  it  has  long  been  recognized  that  computers  can 
be  used  to  help  make  decisions  (Teates,  1986).  Military  decision 
makers,  although  concerned  with  decisions  of  both  large  and  small 
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consequence,  are  no  less  fallible  in  the  use  of  logic  and  influence  than 
are  other  professionals  in  their  area  of  expertise.  These  human 
foibles  can  be  specifically  mitigated  through  the  proper  design  and 
use  of  decision  support  systems.  In  addition,  military  decision 
makers,  knowing  the  consequences  of  poor  decisions  and  rushed  by 
circumstances  beyond  their  control,  must  often  make  their  decisions 
under  considerable  stress  (Teates,  1986). 

Automated  decision  aids  can  help  deal  rapidly,  consistently  and 
accurately  with  the  complex  interrelationship  of  the  many  variables 
inherent  to  battle  management  (Flynn,  1987).  Expert  systems  on  the 
battlefield  can  provide  both  a  better  organized  understanding  of  the 
battle  situation,  as  well  as  better  control  of  the  resources  available. 
The  application  of  expert  systems  includes  intelligence  analysis, 
battlefield  management,  mission  planning,  assessments,  and  threat 
evaluations  (Wolfe,  1986).  In  any  high  stress,  high  rate  of 
information  situation,  human  decision  makers  can  be  overloaded  and 
become  unable  to  assimilate  and  use  all  of  the  available  data.  This  is 
particularly  true  in  combat  management  where  data  must  be 
evaluated  and  used  effectively  in  managing  weapons  and  making 
tactical  decisions  under  strict  time  constraints.  It  is  clear  that 
automated  decision  aids  would  greatly  enhance  military  command 
and  control,  and  Hopple  has  provided  a  paradigm  to  examine  each 
specific  type  of  decision  situation  to  determine  if  a  decision  aid  is 
needed.  The  framework  is  most  useful  in  determining  which 
functions  would  be  best  performed  by  the  decision  aid. 
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One  of  the  main  functions  of  a  decision  support  system  in 
complex  environments,  such  as  military  command  and  control  and 
industrial  process  control,  is  to  monitor  that  environment  and 
present  a  summarized  interpretation  of  that  data  to  the  decision 
maker  to  assist  in  the  decision  making  process. 

A  decision  support  system  can  be  responsible  for 
finding/sensing,  assimilating  and  integrating  relevant  data. 

However,  the  additional  key  responsibility/role  of  a  decision  support 
system  is  to  interpret  that  data  with  respect  to  the  domain 
tasks/goals  and  more  actively  assist  the  human  in  the  decision 
making  process,  with  the  goal  being  higher  quality  and/or  quicker 
decisions.  Woods  (1986,  p.  172-173)  describes  the  key  role  to  be 
played  by  the  decision  support  system  as  "supporting  the  user  by 
providing  informative  counsel"  such  as: 

(1)  warning  the  user  of  potentially  dangerous  conditions 

(2)  reminding  the  user  of  potentially  relevant  data 

(3)  suggesting  alterative  response  strategies 

(4)  performing  information  processing  tasks 

(5)  enhancing  the  user's  information  extraction,  problem 
structuring,  and  thinking. 

The  situation  posed  in  this  research  falls  into  the  closed 
decision/high  data  quality  combination.  The  decision  aid's  primary 
function  in  this  situation  to  aid  information  processing  (i.e.,  assist  the 
uset  in  assimilating  .and  interpreting  the  input  data).  In  the  case  of  a 
closed  decision  situation,  the  human  knows  the  appropriate  response 
for  each  type  of  input  data  and  thus  does  not  need  assistance  in 
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alternative  generation.  In  the  case  of  Eagle-eye  and  Sharpshooter,  a 
decision  support  system  would  have  the  role  of  facilitating  Eagle- 
eye's  decision  making  ability  (both  accuracy  and  speed)  by  assisting 
in  information  processing.  The  goal  of  this  human-machine  system 
would  be  to  maximize  engagement  of  hostile  aircraft,  as  quickly  as 
possible,  while  preventing  the  engagement  of  friendly  aircraft.  For 
example,  a  decision  support  system  might  assimilate  input  data 
about  an  aircraft  (e.g.,  altitude,  speed,  profile.  Identification  Friend  or 
Foe  response,  and  adherence  to  designated  air  corridors)  provided  by 
battlefield  sensors.  It  would  summarize  and  interpret  that  data,  then 
present  the  summarized  information  to  Eagle-Eye.  The  issues 
involved  in  that  interface  will  be  addressed  in  the  human-decision 
support  system  interface  section.  Before  that  interface  is  addressed, 
the  role  and  responsibilities  of  the  human  in  this  joint  system  will  be 
discussed,  as  well  as  the  major  issues  that  arise  with  this 
introduction  of  automated  decision  assistance. 

Role  and  Responsibilities  of  the  Human 

The  introduction  of  an  automated  decision  aid  alters  the  role 
and  responsibilities  of  the  human  and  introduces  some  critical  issues 
concerning  the  coupling  of  human  and  machine. 

The  role  of  the  human  in  a  decision  aided  environment  is  very 
similar  to  that  of  monitoring/supervising  a  complex,  dynamic  system 
without  aid.  The  primary  change  is  interaction  of  the  human  with  an 
automated  system  which  more  actively  assists  the  human  in  the 
decision  making  process  rather  than  merely  facilitate  access  to  the 
data  as  discussed  in  the  previous  section  on  complex  systems  and  the 
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display  literature  (Mitchell,  1983,  1986,  1987;  Rasmussen,  1981, 
1986;  Coury  and  Pietras,  1989;  and  Coury,  Boulette  and  Smith,  1989). 

Perhaps  the  key  responsibility  of  the  human  in  decision 
making  is  accountability  for  the  final  decision.  Fitter  and  Sime 
(1980)  claim  that  the  computer  is  not  and  should  not  be  capable  of 
interacting  on  an  equal  basis.  The  user  should  be  the  master,  the 
computer  a  tool,  with  the  responsibility  of  the  ultimate  decision 
resting  entirely  with  the  human. 

Other  responsibilities  include  detecting  malfunctions  in  the 
decision  support  system,  deciding  when  the  current  situation  is 
outside  of  the  capabilities  of  the  decision  support  system,  and 
determining  when  to  exclude  the  decision  support  system  from  the 
decision  making  process. 

Introduction  of  a  decision  support  system  raises  some  very 
important  issues.  The  first  one  relates  of  the  human's  confidence  in 
the  decision  support  system's  output.  The  user  of  a  decision  support 
system  is  rarely  incompetent  in  the  problem  domain,  thus  will  be  in 
the  position  to  (at  least  subjectively)  make  an  assessment  as  to  the 
reliability  of  the  output  of  the  decision  support  system.  Woods 
(1986)  claims  that  the  value  of  the  information  provided  by  the 
decision  support  system  degrades  rapidly  if  it  exhibits  a  bias  for  or 
against  possible  events,  even  if  the  decision  support  system  is  a 
sensitive  detector.  This  indicates  that  the  human's  confidence  in  a 
decision  support  system  will  be  affected  by  both  its  sensitivity  and 
its  bias. 
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A  closely  related  issue  concerns  the  ability  of  the  human  to 
discriminate  the  correctness  of  the  decision  support  system’s 
information/output.  Does  the  human  have  the  level  of  expertise  to 
recognize  erroneous  output  or  when  a  situation  is  beyond  the 
decision  support  system's  capabilities? 

The  third  issue  synthesizes  the  first  two  issues.  It  relates  to 
the  human's  perceived  or  actual  authority  to  override  the  decision 
support  system's  output.  In  a  discussion  of  the  changing  roles  and 
responsibilities  of  the  decision  maker  upon  the  introduction  of 
automated  decision  support.  Fitter  and  Sime  (1980)  indicate  a  feeling 
that  automated  systems  can  get  out  of  control.  Their  rationale  is  that 
the  people  responsible  for  its  use  may  not  comprehend  the  workings 
of  the  system  thus  leading  to  their  hesitancy  in  overriding  the 
automated  decision  on  some  or  all  decisions.  Their  contention  was 
described  in  the  portion  of  this  section  which  discussed  the  human's 
responsibilities,  specifically  that  the  automated  assistance  was  a  tool 
to  be  used  by  the  human.  This  will  be  discussed  in  detail  in  the 
human-decision  support  system  interface  section. 

The  fourth  issue  is  the  seriousness  of  the  consequences  of  the 
wrong  decision.  If  the  consequence  of  a  potentially  wrong  decision  is 
very  great,  the  human  may  choose  a  cautious  strategy.  How  such 
strategies  will  affect  the  human's  use  of  decision  support  system  is 
unclear. 

The  fifth  issue  is  the  speed  at  which  the  decision  must  be  made 
and  how  this  affects  the  use  of  the  decision  support  system's  output. 

The  sixth  issue  is  the  potential  loss  of  cognitive  skills.  Decision 
makers  may  decide  that  the  machine  can  solve  all  of  their  problems 
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and  thus  they  may  cease  perfecting  their  problem-solving  skills.  The 
loss  of  cognitive  skills,  either  from  deterioration  or  the  failure  to 
continuously  enhance  (update)  needed  skills  has  been  addressed  as  a 
critical  area  of  attention  when  considering  the  use  of  automated 
decision  support  systems.  Work  in  process  control  emphasizes  that 
knowledge  only  becomes  operationally  effective  if  it  has  occasion  to 
be  used  (DeKeyser,  1986).  Issues  on  loss  of  cognitive  skills  are 
related  to  the  loss  of  perceptual -motor  skills  that  arise  in  the  control 
of  automation.  Will  the  human  be  able  to  function  if  the  system  is 
malfunctioning  or  is  unable  to  provide  assistance  (Hoogevens  Report, 
1976;  Chambers  and  Nagel,  1985;  and  DeKeyser,  1986). 

The  last  issue  is  the  reliability  of  these  advanced  technology 
systems.  As  the  complexity  of  automated  systems  increases,  the 
opportunity  for  system  failure  concomitantly  increases  and 
reliability  decreases  (Boming,  1987).  Borning  also  discusses  some  of 
the  sources  that  can  contribute  to  this  potential  system  failure, 
including:  incorrect  or  incomplete  system  specifications,  hardware 
failure,  hardware  design  errors,  software  design  errors  and  human 
errors.  With  complex,  normally  highly  reliable  systems,  a  failure 
may  be  caused  by  an  unusual  combination  of  problems  from  several 
sources.  He  cites,  two  excellent  examples  in  the  national  defense 
arena  illustrating  the  potentially  disastrous  consequences  of  system 
failures.  The  first  was  in  1960  when  the  warning  system  at  Northern 
Air  Defense  Command  indicated  that  the  United  States  was  under 
massive  attack  by  Soviet  missiles  with  a  certainty  of  99.9%,  when,  in 
fact,  the  early  warning  radar  in  Greenland  had  spotted  the  rising 
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moon.  None  of  the  system  designers  had  anticipated  that  the  system 
would  respond  to  the  moon. 

The  second  event  occurred  in  1980.  The  display  system  at  the 
Command  Post  of  Strategic  Air  Command  in  Nebraska  indicated  that 
submarine  launched  ballistic  missiles  were  headed  toward  the  United 
States.  The  displays  suddenly  cleared.  Shortly  thereafter,  the 
warning  indicated  the  intercontinental  ballistic  missiles  had  been 
launched  toward  the  United  States.  The  cause  was  a  failed  chip  in 
the  computer  that  was  used  as  a  communications  multiplexer  to  send 
messages.  When  the  chip  failed,  the  system  started  filling  in  the 
"missiles  detected"  field  in  the  message  with  random  digits  instead  of 
zeros.  These  two  examples  show  both  software  and  hardware 
failures  that  could  have  resulted  in  a  nuclear  exchange  between  the 
United  States  and  the  Soviet  Union.  These  situations  also  illustrate 
how  the  strengths  of  the  human  decision  maker  compensated  for  the 
weakness  of  the  computer,  and  the  critical  importance  of  the  human 
being  ultimately  responsible  for  the  final  decision  and  response. 

There  are  also  design  and  development  issues  that  will  need  to 
be  addressed  in  the  actual  construction  of  a  decision  support  system. 
These  include  (1)  task  component  analysis,  (2)  knowledge 
acquisition,  (3)  expert  selection,  (4)  problem  solving  strategy  to  be 
used  by  the  system,  (3)  allocation  of  functions  between  automation 
and  the  human,  (6)  knowledge  representation,  and  (7)  the  user 
interface. 

As  explained  in  the  Problem  Statement  section,  the  focus  of  this 
research  is  on  how  the  type  of  information,  presented  to  the  human 
by  the  decision  support  system,  affects  total  system  performance. 
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The  literature  review  clearly  illustrates  the  highly .  interactive  nature 
of  decision  support  systems,  supporting  the  hypothesis  that  the  user 
interface  is  critical  to  the  success  and  acceptance  of  the  decision 
support  system  (Wheeler,  1989).  However,  prior  to  focusing  on  the 
human-decision  support  system  interface,  a  review  of  the  major 
research  issues  in  human-decision  support  system  interaction  is 
warranted,  as  well  as  consideration  of  the  cognitive  demands  placed 
on  the  human  when  interacting  with  a  decision  support  system. 
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The  are  a  number  of  major  research  issues  associated  with  the 
consideration  of  the  design  and  development  of  a  decision  support 
system  and  the  interaction  of  the  system  with  the  user.  Some  of 
those  key  research  issues  are: 

(1)  Determine  if  automated  assistance  is  needed 

(2)  Determine  how  the  aid  will  assist  the  user  (i.e.,  the 
decision  making  tasks  that  will  be  the  focus  of  the  decision  aid) 

(3)  Determine  the  functions  the  decision  support  system  will 
perform  and  the  functions  the  user  will  perform 

(4)  Determine  how  the  decision  support  system  will  present 
information  to  the  use" 

(5)  Determir.v  how  the  human  will  be  affected  by  the  use  of 
automated  assistance  (confidence  in  aid,  propensity  to  override, 
ability  to  discriminate  incorrect  assistance,  strategy  selection,  skill 
loss,  etc.) 
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Mitchell  (1986,  p.  356)  highlights  the  importance  of 
understanding  the  cognitive  components  involved  in  the  decision 
making  task  and  their  impact  on  the  design  of  the  display.  She 
states:  "The  design  of  the  display  is  a  critical  determinant  of 
computer-based  display's  effectiveness  in  real-time  control 
environments.  To  answer  the  questions  of  what  information  to 
display,  when,  and  in  what  form  requires  careful  and  detailed 
representation  about  the  cognitive  needs  of  the  human  controlling 
the  system  (i.e.  the  design  must  be  based  on  a  model  of  human 
decision  making  in  the  system)"  With  the  introduction  of  a  decision 
support  system,  the  cognitive  demands  and  subsequent  stages  of 
decision  making  are: 

(1)  Identification  of  the  state  of  the  system/environment. 

(2)  Determination  of  the  alternatives  (solely  or  with  the 
assistance  of  the  decision  aid)  (usually  has  to  be  accomplished 
quickly). 

(3)  Perception,  integration,  interpretation  and  storage  of 
information  received  and  processed  by  the  decision  aid. 

(4)  Analysis  of  the  time  available  to  make  the  decision. 

(5)  Analysis  of  the  consequences  of  wrong  decision  (i.e.,  the 
effect  of  inappropriate  aid  on  the  acceptance  of  the  decision  aid's 
output). 

(6)  Comparison  of  the  user's  decision  to  the  decision  aid's 
recommendation. 

(7)  If  the  decision  by  the  decision  aid  and  the  user’s  decision 
are  incongruent,  then  a  mechanism  for  resolving  the  conflict  must  be 
present. 
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(8)  Make  the  final  decision. 

(9)  Respond  with  the  appropriate  action. 

With  this  summary  of  the  major  research  issues  and  cognitive 
demands  associated  with  human-decision  support  system  interaction 
in  mind,  recall  that  one  of  the  critical  aspects  of  this  interaction  is  the 
manner  in  which  the  decision  aid  presents  information  to  the  user. 
The  literature  on  that  interface  will  now  be  addressed. 

The  Human-Decision  Support  Sv&tem  Interface 

Designing  systems  to  meet  the  needs  of  users  is  the  goal  of 
human  factors,  and  this  must  be  accomplished  at  both  the  physical 
and  cognitive  interface  (McDonald,  1982).  Unfortunately,  most 
human  factors  guidelines  for  computer  displays  focus  on  the  physical 
interface  to  ensure  that  human  sensory  limits  are  not  exceeded  and 
that  users  can  effectively  access  data.  It  rarely,  if  ever,  addresses 
issues  concerned  with  using  the  computer  displays  to  aid  human 
performance  of  domain  tasks  (Woods,  1985,  p.  325).  States  Woods: 
"This  is  particularly  important  in  complex  environments  such  as 
military  command  and  control  and  process  control  (  e.g.,  nuclear, 
thermodynamic,  and  chemical)  where  the  goal  is  more  than  a  useable 
interface.  The  interface  must  support  effective  performance  in  tasks 
such  as  situation  assessment,  fault  management,  problem  solving, 
planning,  and  decision  making.”  Landauer,  commenting  on  human- 
computer  interaction  in  the  year  2000,  said  "at  least  as  important  as 
new  hardware  or  new  modes  of  interaction,  such  as  voice  or  touch, 
will  be  the  great  expansion  in  the  number  and  kinds  of  cognitive 
tasks  that  will  be  supported"  (Landauer,  1986,  p.  253).  Chechile  et 
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al.  (1989,  p.  32)  said  "we  believe  the  cognitive  quality  is  perhaps  the 
most  important  level  of  analysis  in  considering  the  human 
engineering  of  a  display." 

Focusing  on  the  physical  aspects  of  the  interface  can  be  broken 
down  into  two  subcategories:  concentrating  on  the  ergonomic  aspects 
of  the  system,  or  concentrating  on  the  visual  (perceptual)  aspects  of 
the  information  displayed.  In  the  latter,  the  focus  has  been  on  image 
parameters.  This  includes  such  things  as  contrast,  luminance, 
character  legibility,  and  resolution  (McDonald  et  al,  1982;  Eggleston 
et  al.,  1986).  Although  these  are  indeed  critical  aspects  of  the 
interface. 

However,  if  the  purpose  of  the  display  is  kept  in  mind  (i.e.,  to 
communicate  with  people  with  respect  to  their  knowledge,  goals, 
tasks,  and  situation),  and  Mitchell's  contention  that  the  human 
operator  in  complex  systems  will  act  with  competence  and 
confidence  only  when  needed  information  is  provided  in  a  timely 
and  interpretable  form;  another  critical  area  of  the  interface  becomes 
apparent:  the  cognitive  content. 

Much  work  has  been  done  in  an  effort  to  illustrate  the 
importance  of  the  cognitive  content  and  cognitive  quality  of  the 
interface  on  overall  system  performance.  Eggleston  et  al.  (1986) 
define  the  cognitive  quality  of  the  interface  as  the  relative  ease  or 
difficulty  a  person  has  in  deriving  meaning  from  a  particular 
representation.  This  quality  is  the  cornerstone  of  the  design  of  the 
cognitive  interface,  with  the  focus  on  what  is  "conveyed"  not  merely 
displayed.  The  process  broadly  entails  (1)  obtaining  measures  of 
relevant  cognitive  characteristics  of  the  intended  user's  knowledge 
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representation  for  the  task  domain,  and  (2)  mapping  those  data  to  an 
interface  configuration  (McDonald,  1982).  McDonald  describes  a 
study  simulating  a  menu-based  control  system  that  examined  the 
effects  of  organization  (i.e.,  the  arrangement  of  data  in  informational 
categories)  on  performance.  Multi-dimensional  scaling  and 
hierarchical  clustering  were  used  to  organize  the  subsets.  The  results 
showed  that  there  was  a  benefit  from  the  organization,  with  the 
trend  in  all  dependent  measures  favoring  the  organized  menu.  This 
study  illustrates  the  enhancement  in  performance  when  the  system's 
representation  closely  matches  the  user's  representation. 

Rasmussen  (1981)  conducted  a  cognitive  task  analysis  on 
process  control  operators'  diagnostic  strategies  of  fault  finding 
behavior.  One  of  the  strategies  identified  by  Rasmussen  (called 
topographic  search)  is  performed  by  mapping  of  the  system  against 
normal  or  reference  conditions  through  which  the  extent  of  the 
potentially  bad  field  is  gradually  narrowed  down  until  the  problem 
area  is  identified.  For  this  to  be  effective,  the  operator  needs; 

(1)  a  model  of  the  structure  of  the  system  with  various  levels 
of  abstraction; 

(2)  search  rules  or  heuristics; 

(3)  a  model  of  the  normal  operating  state  of  the  system; 

(4)  relationships  among  data  rather  than  just  the  magnitude 
of  the  variables. 

Rasmussen  claims  that  current  nuclear  power  plant  control 
rooms  do  not  support  these  needs  since  there  is  only  one  level  of 
representation  of  plant  state  (i.e.,  the  operator  must  construct  the 
other  levels  mentally).  The  current  one  measurement/one  indication 
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display  philosophy  does  not  show  relationships  between  the  data, 
requiring  the  operator  to  mentally  integrate  the  data.  The  result  is  a 
mismatch  between  the  demand  for  efficient  diagnosis  and  the 
characteristics  of  the  interface,  thereby  increasing  mental  workload 
and  the  possibilities  for  errors.  The  Three  Mile  Island  accident  is  an 
excellent  example  of  the  need  for  emphasizing  the  cognitive  aspects 
of  the  interface.  The  investigation  of  that  accident  revealed  that  the 
necessary  information  was,  in  general,  physically  available,  but  not 
operationally  effective.  The  operators  could  not  integrate  the 
separate  pieces  of  information  (data)  to  make  the  correct  decisions 
(Joyce  and  Lapinsky,  1983).  Rasmussen's  model  suggests  that  the 
display  should  match  a  useful  mental  model  of  the  functional 
structure  of  the  system  at  the  appropriate  level.  Such  a  display 
would  greatly  reduce  the  information  processing  required  by  the 
human  in  preparation  of  making  a  decision. 

There  are  some  key  problems  associated  with  data  displays 
that  neglect  the  cognitive  capacities  of  humans  discussed  earlier. 

Two  of  the  factors  that  have  potentially  devastating  effects  on 
performance  are  (1)  getting  lost,  and  (2)  cognitive  tunnel  vision 
(Woods,  1984). 

Getting  lost  means  that  the  user  does  not  have  a  clear 
conception  of  functional  relationships  within  the  system.  This  results 
in  inefficient  and  incomplete  utilization  of  display  system  data 
resources.  A  study  by  Robertson,  McCracken,  and  Newell  (1981) 
points  out  that  the  "getting  lost"  phenomenon  occurs  even  in  systems 
which  provide  display  selection  mechanisms  in  accordance  with 
accepted  human  factors  guidelines. 
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Cognitive  tunnel  vision  (Moray,  1981)  occurs  when  the  user’s 
attention  is  locked  on  a  subset  of  variables  to  the  exclusion  of  all 
others.  Focusing  attention  on  a  subset  of  variables  can  lead  to  a 
decrement  in  performance,  especially  when  state  identification 
requires  integrating  data  from  several  sources. 

In  an  attempt  to  determine  if  a  summary  would  prevent  the 
user  from  "getting  lost,"  Engel,  Andriessen,  and  Schmitz  (1983)  used 
a  cognitive  engineering  approach  that  provided  an  overview  of  the 
display  structure  as  well  as  summary  status  data  to  improve  total 
system  performance.  With  this  approach,  the  user  did  not  have  to 
remember  or  construct  a  mental  model  of  the  data  structure.  For  the 
data  summary  to  be  effective,  however,  the  relationships  that  are 
important  to  the  user's  task  must  be  portrayed.  Merely  summarizing 
data  is  insufficient.  For  example,  user  performance  suffered  in  an 
example  from  Holnagel  (1981)  because  the  summary  did  not  portray 
the  important  functional  relationships. 

As  an  interim  summary,  the  physical  guidelines  on  the  design 
of  computer  displays  generally  attempt  to  ensure  that  human 
sensory  limits  are  not  exceeded.  The  implicit  assumption  is  that  if 
the  user  can  successfully  process  system  data,  then  s/he  will  find, 
integrate,  and  interpret  all  of  the  relevant  data  at  the  correct  time. 
However,  problems  such  as  "getting  lost"  and  "cognitive  tunnel 
vision"  demonstrate  that  the  successful  processing  of  data  does  not 
guarantee  successful  information  extraction  and  optimal  task 
performance  (Woods,  1984). 

The  analysis  of  what  makes  data  informative  shows  that  the 
question  of  what  constitutes  a  good  display  is  directly  linked  to  what 
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the  human  user  needs  to  know  in  order  to  accomplish  the  domain 
task.  In  other  words,  a  good  display  can  not  be  constructed  or 
judged  unless  one  knows  what  the  display  should  communicate  to 
the  user  (Woods,  1985). 

The  implications  of  this  concept  of  the  cognitive  quality  of  an 
interface  for  human-decision  support  system  interaction  should  now 
be  very  clear.  Once  the  decision  support  system  assimilates  and 
interprets  all  of  the  system/environmental  data  and  is  prepared  to 
present  information  to  the  decision  maker;  a  determination  must  be 
made  on  the  type  of  information  the  decision  aid  present  to  the 
decision  maker.  The  decision  aid  must  communicate  with  the  user 
with  respect  to  the  user's  knowledge/mental  model  of  the  situation, 
goals,  and  tasks,  as  well  as  providing  information  closest  to  that 
needed  by  the  human  to  make  the  decision?  Boming  (1987)  cites  a 
comment  made  by  a  defense  industry  manager  stressing  this  point, 
"the  most  challenging  information  problem  in  the  modem  C3I 
systems  is  the  merging  of  diverse  data  into  a  single,  coherent 
representation  of  the  tactical,  operational,  or  strategic  situation." 

Woods  (1986)  claims  that  the  key  to  effective  application  is  a 
joint  human-machine  cognitive  system.  He  uses  the  term  "cognitive 
coupling"  to  emphasize  the  closeness  that  is  imperative  in  the  design 
of  the  human-computer  interface.  He  proposes  a  design  focus  that  is 
(1)  problem  driven  (as  opposed  to  technology  drive),  (2)  involves  a 
detailed  task  analysis  of  the  cognitive  components  of  the  task,  and 
(3)  is  centered  on  the  joint  cognitive  system.  He  claims  that 
improved  joint  system  performance  depends  on  a  shift  in  emphasis 

t 

away  from  a  display  of  the  data  as  signals  towards  a  focus  on 
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communicating  the  meaning  of  the  data.  He  hypothesizes  that  a 
decision  support  system  should  act  as  an  informative  counsel,  not 
making  or  recommending  solutions,  but  aiding  the  user  in  the 
process  of  reaching  a  decision.  He  further  states  that  if  the  decision 
aid  provides  some  form  of  recommended  solution,  the  user  should 
still  be  ultimately  responsible  for  the  outcome. 

Smith  (1981)  claims  that  work  in  the  area  of  person-machine 
systems  has  established  that  a  machine  locus  of  control  can  have 
strong,  negative  effects  on  the  user  and  thus  total  system 
performance.  Fitter  and  Sime  (1980)  also  point  out  that  problems 
with  user  acceptance  are  often  symptoms  of  underlying  deficiencies 
in  the  cognitive-coupling  between  the  human  and  the  machine. 

Sorkin  and  Woods  (1985)  view  a  joint  cognitive  system  as  two 
decision  makers  in  series.  In  the  first  stage,  the  automated 
subsystem  makes  a  decision  about  the  state  of  the  underlying 
process.  In  the  second  stage,  when  alerted,  the  human  uses  this 
evidence  and  summary  analysis  in  making  the  ultimate  decision. 
Sorkin  and  Woods  claim  that  joint  systems  can  be  enhanced  or 
degraded  based  on  the  interaction  of  the  subsystems  of  human  and 
machine.  The  value  of  the  output  from  the  automated  subsystem  is 
best  considered  as  information  or  evidence  to  be  used  by  the 
decision  maker,  rather  than  a  solution  to  be  accepted  or  rejected. 

When  a  decision  aid  produces  a  recommended  solution,  the 
human  user  remains  responsible  for  the  outcome.  The  user,  of 
course,  has  the  authority  to  override  the  machine,  but  this  type  of 
“cognitive  coupling”  has  strong  negative  implications.  Since  the 
user’s  only  option  is  to  accept  or  reject  the  recommendation,  there  is 
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a  great  danger  of  a  responsibility/authority  double  bind.  This  is  a 
situation  states  Woods  (1986)  where  the  user  either  always  rejects 
the  recommendation  (perhaps  by  finding  or  creating  grounds  for 
decision  aid  unreliability)  or  the  user  totally  abrogates  his  decision 
responsibility  (i.e.,  the  user  does  not  override  the  decision  aid’s 
recommendation,  regardless  of  the  circumstances).  This 
responsibility/authority  double  bind  has  been  observed  with  non-AI 
decision  aids  that  output  solutions  (Fitter  and  Sime,  1980)  and  with 
increases  in  control  automation  that  fail  to  address  the  operator’s 
new  role  as  supervisor  of  the  automated  resources  (Hoogovens 
Report,  1976). 

The  social  psychology  literature  on  human-human  advisory 
interactions  also  illustrates  the  negative  effect  of  providing  a 
recommended  solution  to  a  problem.  Coombs  and  Alty  (1980)  found 
that  unsatisfactory  human-human  advisory  encounters  occurred 
when  the  advisor  offered  a  solution  with  little  or  no  feedback  about 
how  the  problem  was  solved.  On  the  other  hand,  in  more  successful 
encounters,  control  of  the  interaction  was  shared  in  the  process  of 
identifying  the  important  facts  and  using  them  to  ultimately  solve 
the  problem.  Kabanoff  (1985)  states  that  people  are  sensitive  to  and 
protect  their  roles  in  groups  or  dyads.  Any  behavior  on  the  part  of 
another  which  fails  to  recognize  this,  be  it  rational  or  not,  is 
threatening.  Kabanoff  also  reiterates  a  key  idea  in  Brehm's 
Reactance  Theory  which  suggests  that  people  will  act  or  change  their 
attitudes  in  a  way  opposite  to  that  desired  by  an  influence  agent  if 
they  perceive  that  their  freedom  of  choice  or  autonomy  is  threatened 
by  the  influence  agent.  The  parallels  between  the  characteristics  of 
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unsatisfactory  human-human  advisory  encounters  and  the 
characteristics  of  a  joint  cognitive  system  are  evident  and  would 
indicate  that  similar  negative  aspects  will  be  seen  in  the  interaction 
between  a  human  and  a  decision  aid.  Given  the  danger  of  a 
responsibility/authority  double  bind  and  the  negative  effects 
described  in  the  social  psychology  literature,  a  decision  support 
system  that  offers  solutions  rather  than  informative  counsel  may  not 
be  an  effective  model  for  decision  aiding. 

There  are  various  types  of  information  that  a  decision  support 
system  can  present  to  the  decision  maker.  Some  of  these  are: 

(1)  historical,  (2)  predictive,  (3)  status,  and  (4)  a  recommendation 
(sometimes  referred  to  as  advisory  or  command  information).  '  The 
previous  discussion  on  the  joint  human-machine  cognitive  system 
strongly  suggests  that  the  overall  system  performance  is  enhanced 
when  the  information  displayed  by  the  decision  support  system  is 
presented  as  evidence  rather  than  a  solution/recommendation  to  be 
accepted  or  rejected.  This  argument,  played  a  large  role  in  the 
decision  of  which  types  of  information  to  investigate  in  this  research. 
Recall  that  the  focus  of  this  research  is  to  investigate  the  effects  of 
the  type  of  information  presented  by  the  decision  support  system  on 
human  decision  making.  In  particular,  this  research  is  concerned 
with  determining  how  performance  is  affected  when  a  decision 
maker  is  presented  with  status  information  (evidence), 
recommendation  information  (solution),  or  both  types  of  information 
in  a  target  engagement  task. 
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If  the  goal  then  is  to  "fit  the  system  to  the  user,"  then  systems 
must  be  designed  in  such  a  way  that  the  number  of  errors  committed 
by  the  users  is  reduced,  and  the  amount  of  time  required  to 
accomplish  the  task  is  minimized.  "Fit"  in  the  context  of  this  research 
focuses  on  the  meaning  (content)  of  the  information  presented  by  the 
decision  aid  in  its  attempt  to  provide  assistance  to  the  decision 
maker. 

In  the  scenario  facing  Eagle-eye  and  Sharpshooter,  a  decision 
aid  could  potentially  increase  the  accuracy  of  the  decision  and  reduce 
the  amount  of  time  required  to  make  the  correct  decision.  The  role 
of  the  decision  support  system  would  be  to  assist  Eagle-eye  in 
processing  information  by  assimilating  and  interpreting  input  data 
associated  with  aircraft  parameters.  The  key  issues  in  this  research 
are:  (1)  to  determine  if  a  decision  aid  will  enhance  performance  in 
the  type  of  task  facing  Eagle-Eye  and  Sharpshooter,  and  (2)  to 
determine  the  best  type  of  decision  information  to  be  presented  to 
Eagle-eye.  Should  it  present  an  aircraft  status  (i.e.  friendly,  hostile  or 
unidentified)  or  a  recommendation  (i.e.  fire  or  no  fire)  or  maybe 
both? 

Based  on  this  consideration  of  human  interaction  with  a 
decision  support  system  and  the  analysis  of  the  cognitive 
components  of  the  task,  there  is  also  an  area  of  cognitive  psychology 
research  that  needs  to  be  discussed.  In  analyzing  the  cognitive 
components  of  the  task  in  this  research,  a  pattern  emerges  that 
appears  similar  to  the  cognitive  components  of  the  tasks  in  the 
semantic  priming  research.  The  literature  on  that  research  will  now 
be  addressed. 


Priming 

The  concept  of  priming  arose  from  the  Spreading  Activation 
Model  (Collins  and  Loftus,  1975),  a  semantic  network  model 
emphasizing  the  relationship  among  concepts.  The  Spreading 
Activation  Model  assumes  that  when  a  concept  is  processed, 
activation  spreads  out  along  the  paths  of  a  network,  with 
effectiveness  decreasing  as  it  travels  outward.  The  idea  of  activation 
spreading  throughout  a  semantic  network  of  interconnected  concepts 
provides  a  clear  picture  of  semantic  relations  among  concepts.  The 
success  of  this  model,  however,  depends  on  how  well  it  can  account 
for  experimental  results.  One  such  result  is  the  effect  of  semantic 
priming.  Priming  occurs  when  a  decision  about  one  concept  affects  a 
subsequent  decision  about  another  concept.  The  basic  paradigm 
involves  the  use  of  primes  in  certain  tasks  to  determine  whether 
performance  has  been  enhanced  usually  in  terms  of  reduction  in 
response  times. 

Evidence  of  the  priming  effect  was  found  in  the  lexical  decision 
task  (Meyer  and  Schvaneveldt  (1971,  1976).  In  their  research, 
subjects  were  required  to  judge  whether  a  string  of  letters  formed  a 
word.  Some  of  the  strings  were  meaningful  words  (e.g.,  butter)  and 
some  were  not  (e.g.,  nart).  Each  trial  consisted  of  a  pair  of  the 
strings,  the  second  string  presented  immediately  after  subjects  made 
decisions  about  the  first  string,  and  the  measured  variable  was 
response  time  The  most  interesting  results  arose  when  the  two 
strings  were  both  words.  If  the  two  words  were  semantically 
related,  the  subjects  were  faster  in  verifying  that  the  second  string 
was  a  word  than  if  the  two  words  were  unrelated.  For  example, 
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subjects  verified  that  butter  was  a  word  faster  if  it  was  preceded  by 
bread  than  by  nurse.  This  supported  the  idea  that  the  processing  of 
one  word  appeared  to  facilitate  processing  of  a  related  word.  Meyer 
and  Schvaneveldt  argued  that  identification  of  the  first  word  caused 
activation  to  spread  from  the  word’s  location  in  the  lexicon  to  the 
location  of  the  corresponding  concept  in  the  semantic  memory,  and 
then  through  the  semantic -memory  network  to  the  locations  of 
related  words.  When  the  second  string  was  presented,  processing 
already  had  a  “headstart.”  That  is,  the  excitation  from  the  first  word 
reduced  the  amount  of  additional  activation  needed  to  reach  the 
decision  criteria.  As  a  result,  less  processing  of  the  second  word 
string  was  required  before  response  could  be  made,  and  it  was  made 
more  quickly. 

Further  evidence  for  the  priming  effect  was  found  in  a  study 
by  Neely  (1977).  In  this  study,  subjects  made  a  lexical  decision 
about  a  string  of  letters  that  was  presented  following  a  prime  word. 
When  the  prime  word  was  the  name  of  a  category  and  the  letter 
string  was  a  word  that  was  a  category  member,  responses  appeared 
to  be  automatically  facilitated.  This  suggests  that  processing  of  the 
category  name  (the  prime)  spread  activation  to  its  members.  In 
general,  these  .  two  studies  illustrated  that  when  subjects  make  two 
consecutive  semantic  decisions,  response  time  is  facilitated  if  those 
decisions  are  made  about  related  items.  This  suggests  that  the  initial 
semantic  processing  spreads  activation,  which  speeds  later 
processing. 

The  usefulness  of  priming  in  a  problem  solving  task  has  also 
been  studied.  Dominowski  and  Ekstrand  (1967)  reported  that  prior 
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exposure  to  the  solution  word  or  to  an  associate  of  the  solution  word 
decreased  solution  time  in  an  anagram  solving  task.  Seidenstadt 
(1982)  also  conducted  research  on  priming  in  a  problem  solving 
scenario.  He  tested  121  subjects  on  12  single-solution  anagrams. 

Each  anagram  had  either  a  category  label  prime,  list-item  prime,  or 
no  prime.  Compared  with  the  no  prime  condition,  both  of  the  other 
priming  conditions  produced  significant  reductions  in  solution  time. 
These  results  suggest  that  the  priming  effect  can  be  found  in 
problem  solving. 

The  priming  effect  is  important  to  decision  aiding  for  the 
following  reason.  Since  the  purpose  of  a  decision  aid  is  to  present 
information  to  a  decision  maker  to  enhance  performance,  decision 
aiding  is  then  analogous  to  priming.  Subjects  in  this  research  will 
likewise  be  required  to  make  a  decision  about  a  related  concept  that 
is  presented  following  various  types  of  primes  (information  from  a 
decision  aid).  The  cognitive  components  will  include  the  processing 
of  a  prime  (decision  information)  followed  by  a  required  decision 
about  a  concept  related  to  the  prime.  Thus,  the  cognitive  components 
of  the  decision  making  task  in  this  research  bears  striking  similarity 
to  the  word  recognition  and  problem  solving  tasks  in  the  priming 
literature.  Since  the  cognitive  task  structure  in  this  research  is 
analogous  to  that  in  the  earlier  priming  studies,  the  basic  priming 
experimental  paradigm  will  be  used  to  investigate  the  effects  of 
priming  on  a  decision  aid  assisted  decision  making  task.  And,  if  the 
performance  of  the  decision  maker  can  be  enhanced,  what  form 
should  this  priming  take  (i.e.  what  should  the  content  of  the  priming 
information  be)? 
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CHAPTER  3 

RESEARCH  RATIONALE  AND  OBJECTIVES 


As  evidenced  in  the  literature  review,  there  are  a  number  of 
important  research  issues.  The  priming  research  has  shown  that 
priming  an  individual  can  have  a  positive  affect  on  task  performance 
in  the  realm  of  word  recognition  and  anagram  problem  solving.  One 
of  the  objectives  of  this  research  is  to  determine  if  a  similar  effect 
can  be  found  in  a  more  complex  decision  making  task.  The  human- 
machine  research  has  also  shown  that  the  interaction  between  the 
human  and  the  system,  especially  with  respect  to  the  way  in  which 
.  the  information  is  displayed,  plays  a  large  role  in  the  performance  of 
a  task  involving  a  human  and  an  automated  decision  aid.  A  second 
objective  of  this  research  is  to  determine  the  most  effective  content 
of  that  interface  in  a  more  complex  decision  making  environment. 
Specifically,  this  research  investigates  the  effect  of  the  type  of 
information  (semantic  content)  presented  by  a  decision  aid  on  the 
performance  (in  terms  of  accuracy  and  speed)  of  the  human. 

Computer  technology  is  emerging  at  a  far  greater  rate  than  the 
ability  of  human  factors  research  to  rigorously  evaluate  its 
effectiveness.  The  reliability  and  effect  on  operator  performance  and 
workload  of  such  technologies  are  critical  issues  where  human 
decision  making  is  time-constrained  and  the  consequences  of  human 
error  may  be  very  expensive  or  even  catastrophic  (Mitchell,  1987). 
With  the  increasing  commonality  in  the  use  of  automated  decision 
aids  and  thus  the  emergence  of  joint  human-machine  cognitive 
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systems,  a  determination  must  be  made  concerning  the  way  in  which 
that  critical  automated  assistance  is  presented  to  the  human. 

The  proposed  experiment  is  ah  attempt  to  do  that  in  a  complex 
decision  making  environment  (i.e.,  aircraft  engagement)  where  the 
decision  must  be  made  quickly  and  there  is  a  high  cost  of  an  error. 

The  global  research  objective  is  to  improve  the  understanding 
of  human  decision  making  in  demanding  applications  such  as  process 
control  and  military  command  and  control  systems.  The  specific 
objective  of  this  research  relate  to  investigating  the  effect  of  decision 
aiding  on  decision  making  performance.  Performance  refers  to  a 
person’s  ability  to  quickly  and  accurately  process  information  and 
make  a  decision.  The  experimental  issues  of  importance  to  this 
research  are: 

(1)  Determine  the  effectiveness  of  decision  aiding  in 
facilitating  both  more  accurate  and  quicker  decisions 

(2)  Determine  which  type  of  decision  information  is  most 

effective  X 

The  approach  will  be  to  present  scenarios,  similar  to  the  one 
facing  Sp4  Eagle-eye,  to  an  experimental  subject.  Subjects  will  be 
required  to:  (1)  learn  a  set  of  friendly  aircraft,  a  set  of  hostile 
aircraft,  and  two  sets  of  engagement  decision  rules;  (2)  visually 
identify  a  target  stimulus  (aircraft);  (3)  decide  whether  or  not  to  fire 
at  the  aircraft  in  accordance  with  the  decision  rules;  and  (4)  respond 
as  quickly  as  possible. 
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Once  training  and  baseline  performance  measures  have  been 
obtained,  subjects  will  be  presented  with  assistance  from  a  decision 
aid  prior  to  the  presentation  of  an  aircraft  for  identification  and  prior 
to  the  decision  of  whether  to  shoot  or  not.  The  assistance  will  vary  in 
type  and  serve  as  both  the  priming  information  and  the  automated 
assistance.  The  accuracy  of  the  decision  aid  will  be  set  to  96%  (a  pilot 
study  indicated  that  every  subject  quit  using  the  decision  aid  totally 
after  approximately  30-40  of  the  216  trials  when  the  accuracy  level 
was  91%).  By  designing  the  task  this  way  and  presenting 
information  from  a  decision  aid,  this  experiment  will  be  able  to 
determine  the  effect  of  priming  in  a  decision  making  task  and 
determine  which  type  of  decision  aiding  information  is  most 
effective. 
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CHAPTER  4 
EXPERIMENT 

Decision _ Strategies 

The  are  a  number  of  possible  ways  that  this  task  can  be 
accomplished,  when  the  subject  is  presented  with  the  priming 
information.  In  order  to  address  the  hypotheses  of  this  research, 
these  decision  strategies  that  can  be  used  by  the  subject  must  first 
be  defined.  The  predictions  of  this  research  will  be  based  on  these 
strategies.  These  decision  strategies  were  developed  by  examining 
the  cognitive  components  of  the  task,  coupled  with  the  type  of 
decision  aiding  information  presented. 

The  first  condition  is  one  where  the  subject  is  primed  with  the 
status  (S)  information.  Status  information  is  defined  as  friend, 
hostile,  or  unidentified. 

The  strategy  for  status  (S)  information: 

-read  status  information 

-identify  target  (with  focused  attention,  {primed}) 

-make  engagement  decision 
-respon.d 

The  second  condition,  when  the  subject  is  primed  with  the 
recommendation  information,  is  a  bit  more  complicated.  Three 
different  possible  strategies  are  available. 
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The  first  one,  labeled  Ra,  is: 

•read  recommendation 

-map  (translate)  recommendation  to  possible  status 
-identify  target  (again  with  focused  attention) 

-make  engagement  decision 
-respond 

The  second  strategy  (Rb)  in  this  condition  is: 

-read  recommendation 
-store  recommendation 
-identify  target 

-make  preliminary  engagement  decision 
-compare  to  recommendation  from  decision  aid 
-make  final  engagement  decision 
-respond 

The  third  strategy  (Rc)  in  this  condition  is: 

-read  recommendation 
-see  target  (maybe  identify) 

-respond  immediately  with  the  decision  aid's  recommendation 

The  third  condition,  when  the  subject  is  primed  with  both 
status  and  recommendation  information  also  has  three  possible 
strategies. 


39 


The  first  one  (S/Ra)  is: 

-read  information  (and  ignore  recommendation) 

-identify  target  (with  focused  attention  from  status  info) 

-make  engagement  decision 
-respond 

The  second  strategy  (S/Rb)  in  this  condition  is: 

-read  S/R  information  (and  store  recommendation) 

-identify  target  (with  focused  attention  from  status  info) 

-make  preliminary  engagement  decision 
-compare  to  recommendation  from  decision  aid 
-make  final  engagement  decision 
-respond 

The  third  strategy  (S/Rc)  in  this  condition  is: 

-read  S/R  information  (and  ignore  status) 

-see  target  (maybe  identify) 

-respond  immediately  with  decision  aid's  recommendation 

The  control  group  will  not  receive  any  priming  and  be  labeled 
NP  for  no  priming  information. 

The  hypotheses,  which  are  based  on  these  strategies,  the 
literature  on  priming,  and  the  literature  on  human-machine  cognitive 
systems,  are  presented  below. 
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Predictions 


Accuracy 

Strategies  Rcand  S/Rc  will  have  the  lowest  accuracy  due  to 
the  blind  adherence  to  the  decision  aid.  An  assumption  here  is  that 
the  subjects  would  be  better  than  96%  with  no  assistance.  Subjects 
utilizing  this  strategy  will  systematically  choose  the  incorrect 
decision  whenever  the  decision  aid  recommends  the  incorrect 
decision.  The  no  priming  information  (NP)  condition  would  expect  to 
have  the  next  best  accuracy.  Their  accuracy  will  be  strictly  based  on 
their  skill  in  identifying  the  aircraft  and  choosing  the  correct 
engagement  decision,  based  on  the  weapon  control  mode  they  are  in. 
The  status  (S)  condition,  Ra  and  S/Ra  strategies  will  be  slightly  more 
accurate.  They  will  benefit  from  the  decision  aid  while  still  being 
able  to  catch  the  incorrect  information  from  the  decision  aid.  The 
most  accurate  would  likely  be  the  Rb  and  S/Rb  strategies,  due  to  the 
comparison  step  in  their  decision  making.  They  will,  in  a  sense,  be 
double  checking  their  decision  against  the  one  presented  by  the 
decision  aid  prior  to  making  their  response.  The  equation  illustrating 
the  accuracy  hypothesis  is: 

(Rc=  S/Rc)  <  NP  <  (S  =  Ra=  S/Ra)  <  (Rb  =  S/Rb) 

Based  on  the  instructions  “accuracy  is  mandatory  and  speed  is 
crucial,”  accuracy  is  expected  to  be  high  across  all  information  types 
and  strategies.  The  differences  in  accuracy  are  predicted  to  be  very 
small  and  may  not  even  be  significant  except  for  the  Rc  and  S/Rc 
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strategies.  Thus  the  second  set  of  predictions  concerning  response 
time  will  be  critical  in  assessing  the  differences  in  performance 
based  on  type  of  decision  aiding  information. 

Respflnss  .  Time 

Due  to  strategies  Rcand  S/Rc  resulting  in  an  immediate 
response  using  the  decision  aid's  recommendation,  these  strategies 
will  produce  the  fastest  response  times.  Response  times  may  even 
be  lower  than  the  time  it  takes  to  identify  the  aircraft,  due  the 
subjects  sole  reliance  on  and  immediate  response  to  the  decision  aid. 

The  three  strategies  that  take  advantage  of  the  status 
information:  S,  Ra,  and  S/Ra  (explicitly  in  S  and  S/Ra  and  converted 
in  Ra)  will  have  the  next  fastest  response  times.  This  strategy 
benefits  fully  from  the  category  priming  effect.  The  no  prime  (NP) 
condition  will  follow. 

Strategies  S/Rb  and  Rb  will  produce  the  slowest  response 
times.  The  increased  latency  with  these  strategies  will  be  due  to  the 
additional  cognitive  step  of  comparing  their  decision  with  that 
presented  by  the  decision  aid  prior  to  responding.  Strategy  S/Rb 
should  be  slightly  faster  than  Rb  because  of  the  assistance  of  the 
status  information.  The  response  times  should  follow  this  pattern: 

(Rc  =  S/Rc)  <  (S  =  Ra  =  S/Ra)  <  NP  <  S/Rb  <  Rb 
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This  research  will  attempt  to  look  at  not  only  the  overall  effects 
of  the  different  types  of  information,  but  what  strategies  are  utilized 
by  subjects  when  presented  with  the  various  types  of  information 
and  how  this  affects  their  accuracy  and  response  times. 

Sulmcts 

The  thirty  two  subjects  were  volunteers  from  the  University  of 
Massachusetts  community.  Compensation  for  the  60-120  minute 
long  session  was  based  on  a  $5.00  per  hour  rate.  All  subjects  had  no 
previous  military  service.  All  subjects  that  completed  the 
experiment  had  to  meet  a  criteria  level  of  95%  correct  on  the  last  40 
trials  in  Session  1,  Aircraft  Identification.  This  was  to  ensure  that 
there  would  not  be  large  error  rates  based  on  the  failure  to  be  able 
to  identify  the  aircraft.  The  subjects  were  assigned  to  the  condition 
groups  randomly  with  the  caveat  that  they  could  be  assigned  to  a 
certain  condition  to  assure  that  the  groups  were  not  significantly 
different.  In  those  cases  the  assignment  to  condition  was  made 
based  on  the  response  time  of  correct  responses  of  Session  1.  The 
subject  was  placed  into  a  group  which  would  keep  the  groups 

i 

response  time  averages  for  Session  1  as  close  as  possible.  This 
assured  that  there  was  no  difference  in  the  groups'  average  response 
time  in  that  session.  This  assured  that  any  differences  between 
groups  would  be  due  to  information  type  and  not  due  to  the 
differences  between  the  groups. 
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Stimuli 

The  stimuli  for  this  experiment  consist  of  the  graphic 
representations  of  38  aircraft.  The  representations  are  of  the  side 
view  and  are  black  with  white  to  define  internal  features.  The 
stimuli  are  approximately  two  inches  long  and  1/2  inch  tall.  There 
are  two  categories  of  aircraft  type,  rotary  wing  (helicopters)  and 
fixed  wing  (jets).  There  are  25  fixed  wing  and  13  rotary  wing 
aircraft.  The  aircraft  are  divided  into  three  categories  of 
identification;  friendly,  hostile,  and  unknown.  Ten  are  friendly  (four 
helicopters  and  six  jets)  and  ten  are  hostile  (four  helicopters  and  six 
jets).  There  are  18  unknown  aircraft  (five  helicopters  and  13  jets). 
Six  unknown  aircraft  (two  helicopters  and  four  jets)  are  used  for  the 
training  session  on  aircraft  engagement.  The  other  12  unknown 
(three  helicopters  and  nine  jets)  are  used  in  the  two  test  sessions 
(there  are  four  sessions  in  the  experiment) 

The  sequential  presentation  of  the  aircraft  occur  in  a  random 
order  in  both  the  training  and  test  sessions.  Each  subject  will 
undergo  a  total  of  580  trials  (304  training  and  276  test).  All  subjects 
will  receive  the  exact  same  training,  thus  will  see  the  304  trials  in 
the  training  session  in  the  same  order.  The  first  200  trials  in  the 
training  consist  of  ten  replications  of  the  20  friendly  and  hostile 
aircraft.  The  last  104  trials  in  the  training  session  are  broken  down 
into  two  blocks  of  52  trials  in  each  weapon  control  mode  (tight  and 
free).  Of  the  52  trials  in  each  block,  each  friendly,  hostile,  and 
unknown  (the  six  earmarked  for  training)  aircraft  appears  twice. 

In  the  test  condition,  the  subjects  will  see  the  same  set  of  276 
trials;  however,  .for  each  subsequent  subject,  in  a  certain  test 
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condition  (e.g.  status),  the  order  of  the  276  trials  will  be  different 
(randomized).  These  276  test  trials  are  broken  down  into  two 
sessions,  consisting  of  two  blocks  each. 

The  first  test  session,  called  the  Stimulus  Only  session,  consists 
of  60  trials  with  two  blocks  (one  for  each  weapon  control  mode)  of 
30  trials  each.  Within  those  60  trials  are  24  friendly  aircraft  (14  jets, 
10  helicopters),  24  hostile  aircraft  (14  jets,  10  helicopters),  and  12 
unknown  aircraft  (the  12  for  testing)  (9  jets,  3  helicopters). 

The  216  trials  remaining  are  for  Session  Four  (called  the 
Priming  Session),  the  second  test  session.  Those  216  trials  were 
divided  into  two  blocks  (again  one  for  each  weapon  control  mode)  of 
108  trials  each.'  Within  those  108  trials  are  48  friendly  aircraft  (28 
jets,  20  helicopters),  48  hostile  aircraft  (28  jets,  20  helicopters),  and 
12  unknown  aircraft  (nine  jets,  three  helicopters).  Thus  the  totals  for 
that  last  session  of  216  trials  is  96  friendly  aircraft  (56  jets,  40 
helicopters),  96  hostile  aircraft  (56  jets,  40  helicopters),  and  24 
unknown  aircraft  (18  jets,  six  helicopters). 

The  stimuli  were  presented  on  a  Macintosh  Mac  II  workstation 
using  HyperCard.  The  stimuli  used  in  this  experiment  are  presented 
in  Appendix  A. 

Task 

The  subject's  task  is  to  visually  identify  a  target  aircraft,  and 
then,  in  accordance  with  the  engagement  decision  rules 
corresponding  to  the  current  weapon  control  mode,  make  a  decision 
to  shoot  or  not  to  shoot  at  the  aircraft  .  The  response  is  to  be  made 
as  quickly  as  possible  by  pressing  a  key  indicating  that  decision. 
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Procedure 

The  subjects  are  instructed,  in  writing,  as  to  the  task  they  will 
be  performing.  They  are  also  told  that  the  single  subject  with  the 
best  overall  performance  will  receive  a  bonus  of  $25.00.  The  general 
instructions  are  read  to  the  subject.  The  subject  then  begins  by 
studying  cards  of  the  20  aircraft  stimuli  and  the  engagement  decision 
rules  for  30  minutes.  The  engagement  rules  are  as  follows: 

(1)  In  weapon  control  mode  Tight:  Fire  only  at  aircraft 
identified  as  hostile. 

(2)  In  weapon  control  mode  Free:  Fire  at  any  aircraft  not 
identified  as  friendly  (i.e.  hostile  and  unknown). 

The  subject  then  begins  the  computerized  portion  of  the  experiment. 

Session  1; _ Training  on  Aircraft  Identification.  The 

session  consists  of  200  trials.  In  each  trial,  the  subject  will  see  two 
screens  sequentially.  The  first  screen  presents  an  aircraft.  The 
subject  then  decides  whether  it  is  friendly  or  hostile.  The  left  arrow 
key  is  for  Friendly  and  the  right  arrow  key  is  for  Hostile.  The  next 
screen  provides  feedback.  It  shows  the  subject's  decision,  and  the 
correct  answer. 

tabkey  —  Stimulus  — >  arrowkey  —  Feedback  — >  tabkey 

start  trial  friend,  hostile  start  next  trial 

Session  2; _ Training  on  Aircraft  Engagement.  This 

session  consists  of  104  trials  of  engagement  decisions,  52  in  each 
weapon  control  mode  (tight,  free).  In  this  section  when  the  aircraft 
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is  presented,  the  subject  decides  whether  to  shoot  at  it  or  not.  The 
left  arrow  key  is  designated  "No  Fire"  and  the  right  arrow  key  is 
designated  "Fire".  Unknown  aircraft  (i.e.  aircraft  not  learned  as 
friendly  or  hostile)  are  presented  in  this  session.  The  next  screen 
will  again  present  feedback.  This  feedback  screen  will  show  the 
weapon  control  mode  (tight,  free),  aircraft  type  (friend,  hostile  or 
unidentified),  the  subject's  response  and  the  correct  answer. 

tabkey  —  Stimulus  — >  arrowkey  —  Feedback  — >  tabkey 

start  trial  fire,  no  fire  start  next  trial 

Session  3; _ Aircraft  Engagement.  Stimulus  Only.  Session 

3  is  the  first  of  two  test  sessions.  The  task  in  Session  3  is  again 
target  engagement.  The  subject  will  be  presented  with  a  total  of  60 
trials  (30  tight,  30  free)  and  will  respond  with  a  decision  of  Fire  or 
No  Fire.  There  is  no  feedback  between  stimulus  presentations  in  this 
session.  The  screen  displays  "Hit  tabkey  when  ready  for  next 
scenario." 

Ready  for  — >  tabkey  —  Stimulus  — >  arrowkey 

next  scenario  fire,  no  fire 

start  trial  start  next  trial 

Session  4: _ Aircraft  Engagement  With  Priming.  In 

Session  4,  the  subjects  in  the  experimental  groups  will  be  presented 
with  the  priming  information.  The  subjects  viewed  the  screen  as 
long  as  they  wished.  The  information  pertained  to  the  aircraft  to  be 
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identified  on  that  trial.  The  subject  will  press  the  tab  key  and  the 
aircraft  will  be  presented.  The  subject  will  respond  once  again  with 
Fire  or  No  Fire.  There  are  216  trials  in  this  session  (108  tight,  108 
free).  The  control  group  in  Session  4  will  not  receive  the  priming 
information,  but  repeat  the  procedure  of  Session  3  for  their  216 
Session  4  trials. 

Ready  for— >  tabkey—  Priming->  tabkey— Stimulus—  >  arrowkey 
next  scenario  Info  fire,  no  fire 

start  trial  start  next  trial 

The  subjects  will  have  periodic  rest  breaks  as  needed  and  the 
duration  of  the  entire  experiment  should  not  exceed  120  minutes. 

Exp&rimentaL  -Email 

A  between  subjects  design  will  be  used  to  present  each  of  four 
primary  conditions.  The  overall  independent  variable  is  information 
type.  The  four  levels  of  this  treatment  are:  (1)  status  information, 

(2)  recommendation  information,  (3)  status  and  recommendation 
information,  and  (4)  no  priming  information  (control  condition). 
Within  each  condition,  there  are  two  additional  within  subject 
factors:  weapon  control  mode  (tight  or  free)  and  blocks  of  trials. 
Weapon  control  mode  is  blocked  within  subject  and  counter  balanced 
across  subjects.  The  blocks  of  trials  was  also  a  within  subjects  factor 
and  will  assist  in  observing  the  effects  of  practice  and  fatigue. 
Subjects  are  treated  as  random  and  all  other  factors  are  treated  as 
fixed. 
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The  dependent  measures  are:  (1)  accuracy,  which  is  defined  as 
the  proportion  of  correct  responses  and  (2)  the  response  time  to 
make  the  decision.  The  response  time  was  measured  from  the  onset 
of  the  target  aircraft  to  the  subject's  response.  Response  time  for 
only  correct  responses  were  used  in  the  analysis.  Subjects,  during 
the  test  conditions,  will  experience  60  trials  (30  tight,  30  free)  with 
the  stimulus  only  (no  priming  information)  and  216  trials  (108  tight, 
108  free)  with  priming  information.  The  control  group  (no  prime 
condition)  will  complete  these  last  216  trials  similarly  to  the  first  60 
(i.e.,  276  trials  of  stimulus  only). 

Results 


Introduction 

The  results  will  be  presented  in  five  separate  sections.  Those 
sections  will  be  (1)  training  and  stimulus  only  sessions  results,  (2) 
priming  session  results,  (3)  percent  change  results  between  stimulus 
only  and  priming  sessions,  (4)  results  of  a  more  indepth  look  at 
information  type  performance  across  decision  aid  accuracy,  and  (5)  a 
subjective  assessment  of  subject  strategy  selection.  Complete 
ANOVA  tables  are  located  in  Appendix  C. 


Training,  and  Stimulus  Only  Sessions 

The  baseline  measures  obtained  from  these  first  three  sessions 
assured  that  there  was  no  significant  differences  between  groups 
prior  to  the  introduction  of  the  decision  aid.  Table  1  indicates  the 
performance,  in  both  accuracy  and  response  time,  of  the  four  groups 
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for  the  two  training  sessions;  training  on  aircraft  identification  and 
training  on  aircraft  engagement.  The  groups  are  designated  by  the 
type  of  information  they  would  eventually  be  using  in  the  priming 
session.  No  significant  difference  between  groups  in  these  training 
sessions  was  found  in  the  ANOVA  of  these  results.  Those  results 
were  as  follows:  training  identification  accuracy,  F(3,28)=1.33, 
p=.283;  training  identification  response  time,  F(3,28)=.042,  p=.988; 
training  engagement  accuracy,  F(3,28)=1.45,  p=.249;  and  training 
engagement  response  time,  F(3,28)=.24,  p=.868. 

Table  2  indicates  the  accuracy  and  response  time  performance 
of  the  four  groups  for  session  three,  which  tested  the  subjects  on 
aircraft  engagement  without  the  assistance  of  the  decision  aid  (i.e. 
stimulus  only). 

A  repeated  measures  ANOVA  was  conducted  with  one  between 
subjects  factor  (group)  and  one  within  subjects  factor  (weapon 
control  mode).  The  interaction  of  weapon  control  mode  and  group 
was  not  significant  for  either  accuracy,  F(3,28)=.667,  p=.579  or 
response  time,  F(3,28)=.314,  p=.815.  This  lack  of  interaction  allowed 
the  collapsing  across  weapon  control  mode  in  this  session.  Once  again 
no  significant  difference  between  groups  was  observed  in  terms  of 
accuracy,  F(3,28)=.487,  p=.694  or  response  time,  F(3,28)=.910, 
p=.449. 

The  results  to  this  point  in  the  analysis  have  provided  some 
baseline  measures  and  assured  that  there  was  no  significant 
difference  between  the  groups  prior  to  the  introduction  of  decision 
aiding  information  in  Session  Four. 
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Table  1.  Accuracy  and  Response  Times  in  Training 
Identification  and  Training  Engagement  Sessions. 


Information  Type 

Accuracy 

Training  Identification 

Training _ Engagement 

Status 

.99 

.97 

Recommendation 

.99 

.98 

Status&Recommend. 

.99 

.99 

Control 

1.0 

.98 

Information  Type 
Status 

Recommendation 

Status&Recommend. 

Control 


Response  Time  (msec) 
Training  Identification 
1053 
1037 
1049 
1006 


Training _ Engagement 

1279 

1218 

1261 

1129 
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Table  2.  Accuracy  and  Response  Times  in  Stimulus  Only  Session. 


Information  Tvpe 

Accuracy 

Tight 

Free 

Mean 

Status 

.99 

.98 

.985 

Recommendation 

.97 

.97 

.97 

Status&Recommend. 

.98 

.99 

.985 

Control 

.97 

.98 

.975 

Information _ Type 

Response  Time 

Tight 

(msec) 

Elfifi 

M£gn 

Status 

1050 

1084 

1067 

Recommendation 

885 

850 

867 

Status&Recommend. 

995 

934 

964 

Control 

883 

882 

883 

Printing  Session 

Repeated  measures 

ANOVAs  were  then  conducted  on  the 

results  of  the  session  in  which  three  groups  were  assisted  by  a 
decision  aid  (priming  session).  Subjects  nested  in  groups  (based  on 
the  type  of  information  they  received  from  the  decision  aid)  and 
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crossed  by  weapon  control  mode  and  blocks  of  trials.  Separate 
ANOVAs  were  conducted  for  both  accuracy  and  response  time. 

Accuracy.  Table  3  presents  the  accuracy  data  for  the  priming 
session.  It  presents  the  accuracy  data  for  each  group  in  terms  of 
weapon  control  mode  as  well  as  the  mean  accuracy  for  the  whole 
session.  The  main  effect  of  information  type  was  not  significant, 
F(3,28)=.463,  p=.71.  Likewise,  the  main  effect  of  blocks  of  trials  was 
not  significant,  F(3,28)=.723,  p=.541.  The  main  effect  of  weapon 
control  mode  was  significant,  F(l,28)=4.625,  p=.040.  Subjects  were 
less  accurate  in  weapon  control  mode  Free.  It  appears  that  the 
subjects  become  more  prone  to  making  mistakes  when  they  are 
allowed  more  freedom  to  engage  aircraft,  kind  of  a  "trigger  happy” 
effect. 

Table  3.  Accuracy  in  Priming  Session. 

Accuracy 


Information  Type 

light 

Free 

Mean 

Status 

.99 

.97 

.98 

Recommendation 

.96 

.97 

.965 

Status&Recommend. 

.98 

.96 

.97 

Control 

M 

31 

.975 

.9775 

.9675 
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Response  Time.  Table  4  presents  the  response  time  data  for 
the  priming  session.  It  presents  the  response  time  data  for  each 
group  in  terms  of  blocks  of  trials,  as  well  as  the  mean  response  time 
for  the  whole  session.  There  was  no  main  effect  of  weapon  control 
mode,  F(l,28)=2.142,  p=.154  and  no  significant  interaction  between 
weapon  control  mode  and  information  type,  F(3,28)=.035,  p=.991. 
This  allows  the  weapon  control  modes  to  be  collapsed  and  the 
analysis  to  be  conducted  irrespective  of  weapon  control  mode.  The 
main  effect  of  information  type  was  not  significant,  F(3,28)=5.04, 
p=.683,  however,  there  was  a  main  effect  of  blocks  of  trials  with 
F(3,84)=8.74,  p=.0001. 

Table  4.  Response  Times  in  Priming  Session. 

Response  Time  (msec) 


Information  Type 

1 

Z 

Blfltks 

1 

4 

Mean 

Status 

860 

720 

681 

755 

754 

Recommendation 

715 

663 

613 

645 

659 

Status&Recommend. 

765 

706 

664 

697 

708 

Control 

810 

778 

828 

800 

804 

787.5 

716.8 

696.5 

724.3 

Figure  1  illustrates  the  effect  of  practice  across  trials.  The 
response  times  for  the  subjects,  in  the  three  conditions  in  which  they 
are  assisted  by  the  decision  aid,  decline  for  the  first  three  blocks  of 
trials. 
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Figure  1.  Main  Effect  of  Blocks  of  Trials  in  the  Priming  Session. 


Percent  Change  Results 

Analysis  of  the  percent  change  between  the  Stimulus  Only  and 
Priming  sessions  was  also  conducted  for  response  times.  This 
analysis  compared  the  percent  change  between  the  means  of  the 
Stimulus  Only  data  and  the  means  of  the  Priming  session  data. 

Table  5  depicts  the  results  of  this  analysis.  It  shows  that  the  groups 
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receiving  assistance  from  the  decision  aid  all  demonstrated  a  much 
greater  decrease  in  response  time  than  the  control  group. 


Table  5.  Percent  Change  in  Response  Times  between  Stimulus  Only 
Session  and  Priming  Session. 


Response 

Time 

Information  Type 

Stimulus  Only 

Priming 

Percent  Change 

Status 

1067 

754 

-29.3 

Recommendation 

867 

658 

-24.0 

Status&Recommend. 

964 

708 

-26.6 

Control 

882 

803 

-9.0 

Figure  2  pictorially  represents  this  response  time  decrease 
associated  with  the  use  of  a  decision  aid  with  the  three  groups 
receiving  assistance  from  the  decision  aid  showing  a  much  greater 
decrease  in  response  time  than  the  control  group. 

The  ANOVA  showed  a  significant  difference  in  percent  change 
between  groups,  F(3,28)=3.37,  p=.032.  A  post  hoc  analysis  showed 
that  the  control  group  was  significantly  different  than  each  of  the 
other  groups,  but  none  of  the  groups  receiving  decision  information 
were  significantly  different  from  each  other.  A  descriptive  analysis 
of  the  effects  of  practice  are  presented  in  Appendix  D. 
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Response  Time  (msec) 


Figure  2.  Overall  Percent  Change  between  Stimulus  Only 
Session  and  Priming  Session  for  each  Information  Type. 


Indepth  Analysis  of  Information  Type 

An  analysis  was  conducted  to  determine  if  accuracy  and 
response  times  were  affected  by  the  accuracy  of  the  decision  aid  for 
each  of  the  information  types.  Table  6  shows  these  results.  The 
table  shows  the  accuracy  and  response  time  for  each  information 
type  as  a  function  of  the  level  of  accuracy  of  the  information 
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presented  by  the  decision  aid.  The  three  levels  of  accuracy  are 
correct,  no  help,  and  incorrect.  The  main  effect  of  accuracy  level  of 
the  decision  aid  was  significant  for  both  accuracy,  F(l,21)=7.52, 
p=.012  and  for  response  times,  F(l,21)=5.27,  p=.032.  The  main  effect 
of  information  type  was  not  significant  for  either  accuracy, 
F(2,21)=1.71,  p=.329  or  response  times,  F(2,21)=.037,  p=.964.  The 
interaction  of  information  type  and  accuracy  level  of  the  decision  aid 
was  not  significant  for  either  accuracy,  F(2,21)=1.08,  p=.358  or 
response  times,  F(2,21)=.563,  p=.578. 


Table  6.  Accuracy  and  Response  Times  for  each  Information  Type 
as  a  Function  of  the  Accuracy  of  the  Decision  Aid  . 


Information  Typn 

Accuracy 

Correct 

No  Help 

Incorrect 

Status 

.985 

.979 

.953 

Recommendation 

.973 

.797 

Status&Recommend. 

.985 

.957 

.859 

Information  Tvoe 

Response  Time 

Correct 

No  Help 

Incorrect 

Status 

701 

848 

812 

Recommendation 

622 

992 

Status&Recommend. 

650 

806 

861 
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Figure  3  illustrates  the  accuracy  of  the  subjects  in  each  of  the 
information  types  as  a  function  of  the  level  of  accuracy  of  the 
decision  aid.  Although  not  statistically  different,  a  descriptive 
analysis  appears  to  indicate  a  trend  in  which  status  information 
results  in  the  highest  accuracy  across  all  levels  of  accuracy  of  the 
decision  aid. 


Figure  3.  Accuracy  of  each  Information  Type  as  a  Function 
of  the  Accuracy  of  the  Decision  Aid. 
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Figure  4  illustrates  the  response  times  of  the  subjects  in  each  of 
the  information  types  as  a  function  of  the  level  of  decision  aid 
accuracy.  Again  in  a  descriptive  analysis,  it  appears  that  when  the 
decision  aid  is  correct,  all  three  information  types  are  nearly  the 
same.  When  the  decision  aid  is  incorrect,  status  information  and 
status  and  recommendation  information  have  nearly  identical 
response  times;  however  the  response  times  when  presented  with  an 
incorrect  recommendation  are  higher. 


Figure  4.  Response  Times  for  each  Information  Type  as  a 
Function  of  the  Accuracy  of  the  Decision  Aid. 
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Statistically,  the  results  show  that  the  accuracy  of  the  subject 
decreased  as  the  accuracy  of  the  decision  aid  decreased. 

Descriptively,  the  greatest  decrease  occured  when  subjects  received  a 
recommendation.  The  decrease  was  smallest  when  subjects  received 
status  information,  with  status  and  recommendation  information 
resulting  in  an  intermediate  decrement.  Those  subjects  presented 
with  only  status  information  responded  correctly  in  95%  of  those 
situations  where  the  decision  aid  provided  incorrect  information. 
Subjects  presented  with  status  and  recommendation  information,  and 
recommendation  information  only  responded  correctly  86%  and  80% 
of  the  time  respectively.  It  appears  that  subjects  presented  with 
status  information  were  able  to  recover  more  effectively  when  the 
decision  aid  presented  incorrect  information.  A  post  hoc  analysis  did 
show  that  the  accuracy  of  the  Status  group  and  the  Recommendation 
group,  when  the  decision  aid  was  incorrect,  were  not  significantly 
different,  F(l,21)=2.24,  p=.149.  It  appears  that  a  large  within  group 
variance,  possibly  caused  by  a  small  sample  size,  washed  out  the 
significant  difference  here. 

The  same  pattern  can  be  seen  in  the  response  times. 
Statistically,  the  response  times  increased  as  the  information  from 
the  decision  aid  goes  from  correct  to  no  help  to  incorrect. 
Descriptively,  the  response  times  for  status  information  and  status 
and  recommendation  information  drop  in  a  similar  pattern  and  are 
not  as  affected  by  incorrect  information  from  the  decision  aid  as 
those  receiving  recommendation  information.  An  incorrect 
recommendation  increased  the  response  time  by  almost  30%.  Once 
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again,  it  appears  as  though  a  large  within  group  variance  washed  out 
the  significant  difference  here  as  well. 

An  interesting  tradeoff  further  illustrates  the  potential  hazards 
when  recommendation  information  is  provided.  Two  of  the  eight 
subjects  in  the  recommendation  group  and  one  in  the  status  and 
recommendation  group  chose  a  strategy  that  strongly  adhered  to  the 
recommendation  of  the  decision  aid.  In  this  situation,  the  subject 
responded  with  the  recommended  action  as  soon  as  the  stimulus 
appeared.  This  lead  to  a  steep  decrease  in  response  time,  but  an 
accompanying  rise  in  the  percentage  of  incorrect  responses  when  the 
decision  aid  provided  an  incorrect  recommendation.  Their  percent 
change  in  response  time  from  not  having  assistance  to  being  assisted 
by  the  decision  aid  and  the  corresponding  percentage  of  incorrect 
responses  when  the  decision  aid  was  incorrect  are: 


Table  7.  Speed-Accuracy  Tradeoff  Associated  with  Strategy  Selection. 


Information 

Percent  Change 

Percent 

Incorrect  when 

Subject 

Type 

in  Response  Time 

Decision 

Aid  is  Incorrect 

22 

Recomm 

48% 

50% 

30 

Recomm 

70% 

75% 

19 

Status&Recomm 

40% 

62.5% 

The  average  percentage  incorrect  when  the  decision  aid  was 
incorrect  for  the  other  21  subjects  was  6%.  A  one  way  ANOVA 
showed  that  there  was  a  significant  difference  between  these 
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subjects  and  the  other  21  subjects,  F(2,21)=105.16,  p.<0001.  This 
clearly  shows  that,  when  provided  with  a  recommendation,  some 
people  will  choose  this  blind  adherence  strategy,  and  there  is  a  cost 
associated  with  the  speed  increase,  namely  having  very  little  chance 
of  catching  an  incorrect  recommendation. 

Strategy _ Selection 

An  analysis  of  the  post  experiment  questionnaire  was 
conducted  and  the  strategy  used  by  each  subject  was  determined. 
Strategies  were  determined  by  mapping  the  subject's  comments  to 
the  description  of  the  strategies.  This  analysis  resulted  in  the 
following  breakdown  of  strategy  selection;  recall  there  is  only  one 
strategy  for  those  who  received  status  information.  Descriptions  of 
the  strategies  are  presented  in  Chapter  4. 


Table  8.  Results  of  Subjective  Strategy  Assessment. 


Stralegy  No.  of  Subjects  Accuracy 


s 

8 

.98 

Ra 

1 

.963 

Rb 

4 

.984 

Rc 

2 

.93 

S/Ra 

3 

.97 

S/Rb 

3 

.99 

S/Rc 

1 

.926 

Response  Time  (msec) 
754 
497 
872 
375 

633 

839 

437 
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This  analysis  failed  to  determine  the  strategy  for  the  one 
remaining  subject  in  each  of  the  recommendation  information 
condition  and  the  status  and  recommendation  information  condition. 
This  chart  shows  that  response  times  for  the  Rc  and  S/Rc  strategies 
were  extremely  fast  with  an  accompanying  low  accuracy  whereas  the 
Rb  and  S/Rb  strategies  have  slower  response  times  but  higher 
accuracy. 


Chapter  5 

DISCUSSION  AND  CONCLUSIONS 

Introduction 

The  main  objective  of  this  study  was  to  investigate  the  effect  of 
decision  aiding  on  decision  making  performance.  The  major  research 
questions  were  posed  to  accomplish  this  main  objective.  They  were: 

(1)  Determine  the  effectiveness  of  decision  aiding  in 
facilitating  both  more  accurate  and  quicker  decisions 

(2)  Determine  which  type  of  decision  information  is  most 
effective 

This  research  has  shown  that,  in  this  particular  decision  making 
environment,  decision  aiding  does  have  an  impact  on  the  decision 
making  process.  Decision  aiding  significantly  reduces  the  time  to 
make  a  decision.  All  three  types  of  decision  information  resulted  in 
significantly  faster  decision  making  than  no  decision  information. 

When  examining  the  specific  types  of  information  presented, 
status  information  seems  to  come  to  the  forefront  as  the  information 
type  that  shows  the  most  promise.  It  reduces  the  time  to  make  a 
decision,  provides  strategy  stability,  was  most  favorably  viewed  by 
the  subjects,  as  well  as  quite  possibly  allowing  easier  recovery  from 
faulty  information  from  the  decision  aid.  Recommendation 
information  appears  to  be  the  most  undesirable.  It  does,  on  average, 
reduce  the  time  to  make  a  decision,  but  clearly  has  some  negative 
traits.  The  most  glaring  negative  trait  is  strategy  instability.  One 
main  strategy  does  not  reduce  decision  making  time,  while  the  other 
main  strategy  greatly  reduces  the  decision  making  time  with  almost 
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no  chance  of  catching  an  incorrect  recommendation  before 
responding.  Subjects  also  report  locus  of  control  problems,  as  well  as 
not  particularly  liking  recommendation  information.  Status  and 
recommendation  information  holds  the  middle  ground,  benefitting 
from  the  positive  aspects  of  status  information  and  the  negative 
aspects  of  recommendation  information.  Each  of  the  research  issues 
will  now  be  presented  in  greater  detail. 

The  Effectiveness  of  Decision  Aiding 

Will  the  performance  of  the  decision  maker  be  enhanced  by  the 
presentation  of  information  related  to  the  decision  problem  (i.e.,  is 
decision  aiding  effective  in  facilitating  both  more  accurate  and 
quicker  decisions)?  If  one  considers  the  amount  of  improvement 
that  results  from  providing  decision  aiding  information,  there  is 
evidence  that  a  decision  aid  can  reduce  the  amount  of  time  to  make  a 
decision.  In  this  experiment,  response  times  decreased  from  Session 
Three,  where  the  decisions  were  made  without  the  assistance  of  the 
decision  aid  to  Session  Four,  where  assistance  was  provided  (See 
Table  5  and  Figure  2).  The  decrease  was  significant;  the  groups  who 
were  assisted  by  the  decision  aid  were  approximately  three  times 
faster  than  the  control  group. 

The  potential  benefits  are  predicted  by  the  priming  literature 
and  are  related  to  the  time  to  make  a  decision.  The  research  in 
semantic  priming  (Meyer  and  Schvaneveldt,  1971,1976;  Neely,  1977; 
Seidenstadt,  1982)  suggested  that  decision  aiding  could  enhance  the 
decision  making  process,  at  least  with  respect  to  the  time  required  to 
make  a  decision.  Priming  occurs  when  activation  of  a  concept 
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positively  affects  processing  of  related  concepts.  This  positive  affect 
on  processing  results  in  a  shorter  time  to  make  a  decision.  Since  the 
purpose  of  a  decision  aid  is  to  present  information  to  a  decision 
maker  in  an  attempt  to  enhance  performance,  decision  aiding  is  then 
extremely  analogous  to  priming.  Subjects  in  this  research  were 
likewise  required  to  make  a  decision  about  a  concept  that  was 
presented  following  various  types  of  primes  (from  a  decision  aid). 

An  analysis  of  the  cognitive  components  of  the  tasks  in  the  word 
recognition  and  problem  solving  studies  and  the  decision  making 
task  of  this  research  bear  striking  similarity.  Since  the  cognitive  task 
structure  in  this  research  is  analogous  to  that  in  the  earlier  priming 
studies,  the  semantic  priming  research  would  suggest  that  decision 
aiding  should  enhance  the  decision  making  process,  at  least  with 
respect  to  the  time  required  to  make  a  decision. 

The  decision  support  system  research  predicts  enhanced  total 
system  performance  if  there  is  a  “cognitive-coupling”  between  the 
human  and  the  automated  subsystem.  This  coupling  is  not  display 
content  independent  but  in  fact  depends  greatly  upon  the  manner  in 
which  the  automated  subsystem  presents  information  to  the  human. 
Therefore  the  predictions  are  directly  linked  to  the  type  of 
information  displayed  by  the  decision  aid.  This  will  be  discussed  in 
more  detail  when  the  selection  on  the  best  decision  information  is 
presented. 

The  repeated  measures  ANOVA  of  the  Priming  Session 
appeared  to  indicate  that  the  decision  aid  had  no  effect  on 
performance.  Tables  3  and  4  report  the  accuracy  and  response  times 
for  the  three  groups  with  the  assistance  of  decision  aiding  and  the 
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control  group.  These  results  showed  that  the  decision  aid  did  not 
significantly  affect  the  decision  maker's  ability  to  make  either  more 
accurate  or  quicker  decisions.  If  critical  inquiry  into  this  research 
topic  would  have  stopped  here,  one  would  have  concluded  that 
performance  was,  in  fact,  not  enhanced  enough  to  warrant  the  cost  of 
designing  and  developing  such  a  decision  support  system.  Such  a 
conclusion,  however,  would  have  been  a  mistake,  since  a  slightly 
different  approach  to  the  analysis  uncovered  the  potential  benefits  of 
a  decision  aid. 

Selecting  the  Best  Decision  Information 

Given  that  decision  aiding  is  better  than  no  aid.  The  question  of 
selecting  the  appropriate  decision  aiding  information  becomes 
important.  The  second  research  issue  of  interest  is  related  to  the 
effectiveness  of  each  type  of  information. 

It  has  previously  been  established  that  the  decision  aid 
increased  the  speed  of  decision  making  by  approximately  three  times 
that  of  the  control  group.  However,  that  speed  increase  was  very 
similar  across  all  three  types  of  information.  How  then  can  the  most 
effective  information  type  be  determined?  The  following  three  areas 
were  examined  in  an  attempt  to  determine  the  best  type  of  decision 
information:  (1)  accuracy  and  response  times  for  each  information 
type  at  the  different  levels  of  accuracy  of  the  decision  aid  (correct,  no 
help,  and  incorrect),  (2)  subject's  strategy  selection,  and  (3)  subjects 
protocols. 
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Statistically,  the  results  show  that  the  accuracy  of  the  subject 
decreased  as  the  accuracy  of  the  decision  aid  decreased. 

Descriptively,  the  greatest  decrease  occured  when  subjects  received  a 
recommendation.  The  decrease  was  smallest  when  subjects  received 
status  information,  with  status  and  recommendation  information 
resulting  in  an  intermediate  decrement.  Those  subjects  presented 
with  only  status  information  responded  correctly  in  95%  of  those 
situations  where  the  decision  aid  provided  incorrect  information. 
Subjects  presented  with  status  and  recommendation  information,  and 
recommendation  information  only  responded  correctly  86%  and  80% 
of  the  time  respectively.  It  appears  that  subjects  presented  with 
status  information  might  be  able  to  recover  more  effectively  when 
the  decision  aid  presents  incorrect  information.  A  post  hoc  analysis 
did  show  that  the  accuracy  of  the  status  group  and  the 
recommendation  group,  when  the  decision  aid  was  incorrect,  were 
not  significantly  different,  F(l,21)=2.24,  p=.149.  It  appears  that  a 
large  within  group  variance,  possibly  caused  by  a  small  sample  size, 
washed  out  the  significant  difference  here  (See  Figure  4). 

The  same  pattern  can  be  seen  in  the  response  times. 
Statistically,  the  response  times  increased  as  the  information  from 
the  decision  aid  goes  from  correct  to  no  help  to  incorrect. 
Descriptively,  the  response  times  for  status  information  and  status 
and  recommendation  information  drop  in  a  similar  pattern  and  are 
not  as  affected  by  incorrect  information  from  the  decision  aid  as 
those  receiving  recommendation  information.  An  incorrect 
recommendation  increased  the  response  time  by  almost  30%.  Once 
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again,  it  appears  as  though  a  large  within  group  variance  washed  out 
the  significant  difference  here  as  well  (See  Figure  5). 

A  person's  strategy  in  using  decision  information  is  also  a 
critical  issue,  since  the  choice  greatly  affects  overall  performance  in  a 
task,  especially  with  respect  to  tradeoffs  between  accuracy  and 
speed.  The  analysis  of  the  subjective  assessments  of  subject  strategy 
showed  that  various  strategies  are  indeed  utilized  when 
recommendation  or  status  and  recommendation  information  is 
presented  by  the  decision  aid.  The  means,  for  accuracy  and  response 
time  for  each  of  the  strategies,  were  computed  and  mapped  to  the 
predictions  made  in  Chapter  4.  The  predictions  with  the 
corresponding  strategy  are  presented  below.  The  analysis  showed 
that  when  strategies  were  identified  from  the  questionnaire  and 
associated  with  performance,  there  is  a  very  close  correspondence  to 
the  predictions. 

Accuracy 

(Rc=  S/Rc)  <  NP  <  (S  =  Ra=  S/Ra)  <  (Rb  =  S/Rb) 

.93  .926  .97  .98  .963  .97  .984  .99 

Response  Time 

(Rc  =  S/Rc)  <  (S  =  Ra  =  S/Ra)  <  NP  <  S/Rb  <  Rb 

375  437  754  497  633  803  839  872 
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The  key  point  to  be  stressed  is  the  fact  that  recommendation 
information  and  status  and  recommendation  information  lend 
themselves  to  the  utilization  of  different  strategies,  two  of  which  are 
at  opposite  ends  of  the  speed-accuracy  tradeoff.  Notice  that  the 
response  times  for  the  Rc  and  S/Rc  strategies  were  extremely  fast 
with  an  accompanying  low  accuracy  and  the  Rb  and  S/Rb  strategies 
have  slower  response  times  but  higher  accuracy. 

An  example  of  this  interesting  tradeoff  further  illustrates  the 
potential  hazards  when  recommendation  information  is  provided. 
Two  of  the  eight  subjects  in  the  recommendation  group  and  one  in 
the  status  and  recommendation  group  chose  a  strategy  that  strongly 
adhered  to  the  recommendation  of  the  decision  aid.  In  this  situation, 
the  subject  responded  with  the  recommended  action  as  soon  as  the 
stimulus  appeared.  This  lead  to  a  steep  decrease  in  response  time, 
but  an  accompanying  rise  in  the  percentage  of  incorrect  responses 
when  the  decision  aid  provided  an  incorrect  recommendation.  Their 
percentage  of  incorrect  responses  when  the  decision  aid  was 
incorrect  were  significantly  worse  than  the  rest  of  the  subjects.  This 
clearly  shows  that,  when  provided  with  a  recommendation,  some 
people  will  choose  this  blind  adherence  strategy,  and  the  cost 
associated  with  the  speed  increase  is  that  of  reducing  the  chance  of 
catching  an  incorrect  recommendation. 

This  strategy  analysis  points  to  another  positive  aspect  of 
status  information.  It  is  stable.  Status  information  does  not  lend 
itself  to  dichotomous  strategies  and  unpredictable  performance. 
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Lastly,  some  observations  made  by  the  subjects  shed  some 
light  on  the  way  they  felt  about  working  with  the  decision  aid  in 
general  and  with  the  different  types  of  information  in  particular. 

In  general,  many  subjects  were  skeptical  about  working  with 
the  decision  aid,  and  had  more  trust  in  their  own  judgement.  This 
seems  reasonable  since  they  had  little  experience  in  working  with  a 
decision  aid  and  especially  this  one  in  particular.  Recall  that  the 
actual  accuracy  of  the  decision  aid  was  96%.  Of  the  24  subjects 
working  with  the  decision  aid,  two  thirds  of  them  perceived  the 
accuracy  of  the  decision  aid  as  less  than  it  actually  was,  seven  of  the 
subjects  estimated  the  accuracy  between  60-75%,  and  87%  thought 
they  were  more  accurate  than  the  decision  aid.  In  fact,  87%  of  those 
working  with  the  decision  aid  were  more  accurate  than  the  decision 
aid  in  the  Stimulus  Only  Session.  Tolerance  for  decision  aid  mistakes 
was  extremely  low  and  subjects  lost  confidence  in  the  aid  quickly 
(i.e.  after  only  one  error  or  perceived  error  in  many  cases).  This 
resulted  in  many  subjects  indicating  that  they  would  rather  do  the 
task  themselves.  Finally,  five  subjects  mentioned  that  the  use  of 
color  as  the  presentation  medium  might  be  better  than  having  to 
read  the  words  that  convey  the  information  from  the  decision  aid. 

Also  as  part  of  the  questionnaire  (Appendix  B),  subjects  were 
asked  to  rate  the  way  they  felt  about  being  assisted  by  the  decision 
aid  on  a  scale  of  1  (negative)  to  10  (positive).  The  overall  average 
was  5.11,  with  the  information  type  averages  being:  status  (5.36), 
recommendation  (4.12),  and  status  and  recommendation  (5.87). 

About  the  only  thing  this  shows  is  that,  were  ambivalent  towards  the 
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decision  aid  and  that  recommendation  information  was  “liked”  less 
than  the  other  two. 

The  summarized  comments  by  the  groups  presented  with  the 
different  information  types  follows.  Those  presented  with  status 
information  generally  said  that  the  decision  aid  was  helpful.  Some  of 
the  comments  were:  “It  was  helpful,  it  gave  me  a  headstart  on  what  I 
was  looking  for;”  “the  decision  aid  was  helpful  when  I  was  unsure 
about  the  aircraft,  I  had  no  problems  overriding  the  decision  aid;” 
and  “I  felt  it  was  useful  in  recognition,  but  did  not  rely  on  it.”  The 
group  presented  with  a  recommendation  were  generally  not  very 
pleased.  Many  had  negative  comments.  Some  of  those  comments 
were:  “It  was  more  of  a  hindrance  than  help,  wrong 
recommendations  were  annoying;”  “I  did  not  like  the  decision  aid;”  “I 
felt  it  was  actually  a  handicap,  sometimes  I  didn’t  let  my  own 
knowledge  override  the  decision  aid’s  recommendation;”  “I  was 
frustrated  with  an  already  made  idea  in  my  mind;”  and  “I  didn’t  like 
it,  it  confused  me.  I  like  being  in  control  and  with  the  decision  aid  I 
felt  like  I  wasn’t.”  The  group  presented  with  a  status  and 
recommendation  had  mixed  comments,  which  seem  to  generated 
based  on  what  piece  of  information  the  subject  focused  on.  Some  of 
those  comments  were:  “The  recommendation  made  me  hesitate;”  “I 
ignored  the  recommendation  part  of  the  information  and  used  the 
status  information;”  “I  could  not  ignore  the  decisions  aid’s 
recommendation,  I  usually  pressed  the  key  suggested  by  the  decision 
aid;”  and  “It  was  a  good  reference,  but  I  didn’t  rely  on  it.  I  never 
really  looked  at  the  recommendation  information.” 
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Summing  up  these  comments,  it  appears  the  people  have  a 
hard  time  estimating  the  actual  accuracy  of  the  decision  aid,  although 
they  seem  to  be  able  to  know  they  are  more  accurate  and  can,  for  the 
most  part,  know  the  times  when  the  decision  aid  was  wrong,  many 
times  after  they  made  their  response  however.  People  are  skeptical 
and  even  reluctant  to  trust  computer  assistance  in  tasks  where  they 
are  responsible  for  the  decision  (especially  if  the  aid  is  giving  a 
recommendation).  They  are  intolerant  of  mistakes  and  want  the 
automated  decision  aid  to  be  perfect.  If  it  is  not,  they  would  rather 
do  the  task  by  themselves  so  as  to  not  be  further  confused  by  an 
imperfect  decision  aid.  Those  working  with  the  status  information 
were  generally  positive.  Those  working  with  the  recommendation 
information  were  definitely  not  favorable  towards  the  aid  and  those 
working  with  status  and  recommendation  information  had  mixed 
feelings  toward  the  decision  aid. 

The  analysis  of  the  data  from  this  experiment  indicates  that 
status  information  (i.e.  friend,  hostile,  or  unidentified)  holds  the  most 
promise.  This  is  in  line  with  both  the  priming  research  and  decision 
support  system  literature. 

The  priming  research  in  both  lexical  decision  tasks  (Neely, 

1977)  and  anagram  problem  solving  tasks  (Seidenstadt,  1982) 
showed  that  category  primes  resulted  in  faster  response  times.  In 
this  research,  status  information  is  equivalent  to  a  category  prime, 
because  aircraft  are  members  of  categories  approximately 
corresponding  to  the  status  information.  This  priming  research 
suggests  that  status  information  should  result  in  the  priming  effect. 


positively  affecting  processing  speed  which  should  result  in 
decreased  time  to  make  a  decision. 

In  the  decision  support  literature.  Woods  (1986)  claims  that 
the  decision  support  system  should  act  as  an  informative  counsel.  It 
should  not  be  used  to  make  or  recommend  solutions,  but  to  aid  the 
user  in  the  process  of  reaching  a  decision.  Smith  (1981)  claims  that  a 
machine  locus  of  control  can  have  strong  negative  effects  on  the  user 
and  total  system  performance.  Sorkin  and  Woods  (198S)  state  that 
the  value  of  output  from  an  automated  subsystem  is  best  considered 
as  evidence  to  be  used  by  the  decision  maker,  rather  than  as  a 
solution  to  be  accepted  or  rejected. 

Candustons  and _ Recommendations 

In  conclusion,  this  research  has  shown  that,  in  this  particular 
decision  making  environment,  decision  aiding  does  have  an  impact 
on  the  decision  making  process.  Decision  aiding  significantly  reduces 
the  time  to  make  a  decision.  All  three  types  of  decision  information 
resulted  in  significantly  faster  decision  making  than  no  decision 
information. 

When  examining  the  specific  types  of  information  presented, 
status  information  seems  to  come  to  the  forefront  as  the  information 
type  that  shows  the  most  promise.  It  reduces  the  time  to  make  a 
decision,  provides  strategy  stability,  was  most  favorably  viewed  by 
the  subjects,  as  well  as  quite  possibly  allowing  easier  recovery  from 
faulty  information  from  the  decision  aid.  Recommendation 
information  appears  to  be  the  most  undesirable.  It  does,  on  average, 
reduce  the  time  to  make  a  decision,  but  clearly  has  some  negative 
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traits.  The  fust  is  strategy  instability.  One  main  strategy  does  not 
reduce  decision  making  time,  while  the  other  uiain  strategy  greatly 
reduces  the  decision  making  time  with  almost  no  chance  of  catching 
an  incorrect  recommendation  before  responding.  Subjects  also  report 
locus  of  control  problems,  as  well  as  not  particularly  liking  this  type 
of  decision  information.  Status  and  recommendation  information 
holds  the  middle  ground,  benefiting  from  the  positive  aspects  of 
status  information  and  the  negative  aspects  of  recommendation 
information. 

There  are  some  methodological  issues  that  need  to  be 
addressed  and  as  well  as  the  corresponding  recommendations  for 
future  research  on  this  topic.  These  issues  potentially  indicate  why 
significant  accuracy  and  response  time  differences  may  not  have 
shown  up  in  the  Priming  Session.  The  first  is  sample  size.  In  this 
experiment,  only  eight  subjects  were  assigned  to  each  condition.  This 
was  thought  to  be  enough,  but  in  fact  may  not  be. 

The  second  and  more  serious  issue  centers  around  the 
aidability  determination  discussed  by  Hopple  (1986).  Recall  in  his 
paradigm,  the  decision  situation  variable  focused  on  the  complexity 
of  the  decision  to  be  made  and  the  quality  of  the  data  input.  It 
appears  that  in  this  research,  the  quality  of  the  data  input  was  too 
high.  In  general  the  subjects  were  completing  the  task  so  quickly 
that  the  assistance  from  the  decision  aid  was  helpful,  but  not  an 
appreciable  amount.  According  to  Hopple's  paradigm,  the  high  data 
input  quality  (i.e.  aircraft  stimuli)  resulted  in  a  lower  impact  by  the 
decision  aid  than  predicted.  A  recommended  methodological 
adjustment  would  be  to  degrade  the  data  input  quality  (i.e.  make  it 
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harder  to  identify  the  aircraft).  This  should  increase  the  impact  of 
the  decision  aid.  The  author  is  already  working  on  this  by 
considering  the  possibility  of  systematically  decreasing  the  visibility 
and  size  of  the  stimuli. 

The  third  issue  is  a  also  crucial  one  and  one  that  was  discussed 
in  detail  in  the  section  on  mediating  factors  in  the  decision  process. 

It  deals  with  the  cognitive  strategy  subjects  choose  when  presented 
with  a  certain  type  of  information.  Some  of  these  strategies  have 
dichotomous  results  in  both  accuracy  and  response  time.  When 
results  are  averaged  across  subjects  receiving  the  same  information 
type  (e.g.  recommendation),  but  using  different  strategies,  the  results 
make  it  hard  to  arrive  at  general  conclusions  about  the  overall  effect 
of  that  information  type  without  looking  deeper. 
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APPENDIX  A 
STIMULUS  SET 


Friends 


AH1  Cobra  OH58  Kiowa 


F4  Phantom  F5  Freedom  Fighter 
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Hostile 


MI24  Hind  MI4  Hound 


MI6  Hook  MI8  Hip 


MIG21  Fishbed 


AN12  Cub 
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Unknown 


A4  Skyhawk 


Alphajet 


BO  105 


AV8  Harrier 


CH47  Chinook 


G91  Lightning 
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Gazelle 


Lynx 


IL28  Beagle  SU7  Fitter 


OVIO  Bronco  OV1  Mohawk 
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APPENDIX  B 
QUESTIONNAIRE 


1.  Did  you  use  the  information  from  the  decision  aid? 

2.  If  so  how  much  (in  a  %  please)?  If  not,  why? 

3.  Prior  to  the  use  of  the  decision  aid,  when  I  said  the  decision  aid 
was  very  accurate,  what  did  very  accurate  mean  to  you  in  terms  of  a 
%  of  the  time  you  thought  the  decision  aid  would  give  you  the  correct 
answer  (ie.  how  accurate  did  you  think  the  decision  was  going  to  be)? 


4.  How  accurate  do  you  think  the  decision  aid  was  (in  a  %  please)? 

5.  Did  you  feel  you  were  more  accurate  than  the  decision  aid? 

6.  How  did  you  use  the  information  presented  by  the  decision  aid 
(ie.  what  strategy  did  you  use  to  make  your  decision  to  shoot  or  not 
when  you  had  the  decision  aid  to  assist  you)?  Use  Back  if  needed. 


7.  Rate  the  way  you  felt  about  being  assisted  by  the  decision  aid  in 
this  task,  (circle  the  appropriate  number  and  make  comments  if  any) 

negative  positive 

123456789  10 

comments: 

8.  If  you  could  design  a  decision  aid  to  assist  you  in  this  task,  what 
would  you  have  it  present  to  you?  (check  one). 

_ Status  information  only  (ie.  Friend,  Hostile  or  Unidentified) 

_ Recommendation  information  only  (ie.  Fire  or  No  Fire) 

_ A  combination  of  Statur^and  Recommendation  information 

_ Nothing 

_ Something  else(explain) _ 

9.  Any  other  comments  please.  Use  back  if  needed. 


APPENDIX  C 


ANALYSIS  OF  VARIANCE  TABLES 


Training — Identification  Session  (Accuracy! 


SOURCE 

SUM-OF-SQUARES 

DF 

MEAN-SQUARE 

F-RATIO  P 

INFOTYPE 

0.001 

3 

0.000 

1.333  0.283 

ERROR 

0.004 

28 

0.000 

Training _ Identification  Session  (Response  Time) 


SOURCE 

SUM-OF-SQUARES 

DF 

MEAN-SQUARE 

F-RATIO 

P 

INFOTYPE 

10766.375 

3 

3588.792 

0.042 

0.988 

ERROR 

2366075.500 

28 

84502.696 

Training  Engagement  Session  (Accuracy) 


SOURCE 

SUM-OF-SQUARES 

DF 

MEAN-SQUARE 

F-RATIO 

P 

INFOTYPE 

0.002 

3 

0.001 

1.453 

0.249 

ERROR 

0.012 

28 

0.000 

Training  Engagement  Session  (Response  Time) 


SOURCE 

SUM-OF-SQUARES 

DF 

MEAN-SQUARE 

F-RATIO  P 

INFOTYPE 

106912.625 

3 

35637.542 

0.240  0.868 

ERROR 

4162210.250 

28 

148650.366 

SOURCE 

SUM-OF-SQUARES 

DF 

MEAN-SQUARE 

F-RATIO 

P 

HYPOTHESIS 

0.003 

3 

0.001 

0.487 

0.694 

ERROR 

0.051 

28 

0.002 

Main  Effect,  of  Weapon  Control  Mode 


SOURCE 

SUM-OF-SQUARES 

DF 

MEAN-SQUARE 

F-RATIO 

P 

HYPOTHESIS 

0.000 

1 

0.000 

0.438 

0.513 

ERROR 

0.017 

28 

0.001 

Weapon  ..Control  Mods _ bY  information  Type  Interaction 


SOURCE 

SUM-OF-SQUARES 

DF 

MEAN-SQUARE 

F-RATIO 

P 

HYPOTHESIS 

0.001 

3 

0.000 

0.667 

0.579 

ERROR 

0.017 

28 

0.001 

Stimulus  Only  Session  (Response  Time) 


Main  Effect  of  Information  Type 


SOURCE 

SUM-OF-SQUARES 

DF 

MEAN-SQUARE 

F-RATIO 

P 

HYPOTHESIS 

403609.672 

3 

134536.557 

0.910 

0.449 

ERROR 

4138589.063 

28 

147806.752 

SOURCE 

SUM-OF-SQUARES 

DF 

MEAN-SQUARE 

F-RATIO  P 

HYPOTHESIS 

3953.266 

1 

3953.266 

0.181  0.674 

ERROR 

611140.063 

28 

21826.431 

W.ftap.01L  .Control  ,  Mode _ by  Information  Type  Interaction 


SOURCE 

SUM-OF-SQUARES 

DF 

MEAN-SQUARE 

F-RATIO  P 

HYPOTHESIS 

20554.172 

3 

6851.391 

0.314  0.815 

ERROR 

611140.063 

28 

21826.431 

Erimine  Session _ (Accuracy) 


Main  Effect  of  Information  Type 


SOURCE 

SUM-OF-SQUARES 

DF 

MEAN-SQUARE 

F-RATIO  P 

HYPOTHESIS 

0.010 

3 

0.003 

0.463  0.710 

ERROR 

0.197 

28 

0.007 

Main  Effect  of  Weapon  Control  Mode 


SOURCE 

SUM-OF-SQUARES 

DF 

MEAN-SQUARE 

F-RATIO 

P 

HYPOTHESIS 

0.007 

1 

0.007 

4.625 

0.040 

ERROR 

0.043 

28 

0.002 

Weapon  Control  Mode  by  Information  Type  Interaction 


SOURCE 

SUM-OF-SQUARES 

DF 

MEAN-SQUARE 

F-RATIO  P 

HYPOTHESIS 

0.007 

3 

0.002 

1.645  0.202 

ERROR 

0.043 

28 

0.002 

tin  Effect 

of  Blocks 

SOURCE 

SUM-OF-SQUARES 

DF 

MEAN-SQUARE 

F-RATIO  P 

HYPOTHESIS 

0.003 

3 

0.001 

0.723  0.541 

ERROR 

0.099 

84 

0.001 

Block  bv  Information  Type  Interaction 


SOURCE 

SUM-OF-SQUARES 

DF 

MEAN-SQUARE 

F-RATIO 

P 

HYPOTHESIS 

0.014 

9 

0.002 

1.287 

0.256 

ERROR 

0.099 

84 

0.001 

Weapon  Control  Mode  bv  Block  Interaction 


SOURCE 

SUM-OF-SQUARES 

DF 

MEAN-SQUARE 

F-RATIO 

P 

HYPOTHESIS 

0.006 

3 

0.002 

1.828 

0.148 

ERROR 

0.097 

84 

0.001 

Weapon  Control  Mode  bv  Block  bv  Information  Type 

Ln-Ura.Ui.au 


SOURCE 

SUM-OF-SQUARES 

DF 

MEAN-SQUARE 

F-RATIO 

P 

HYPOTHESIS 

0.008 

9 

0.001 

0.797 

0.620 

ERROR 

0.097 

84 

0.001 

86 


Priming  Session  (Response  Time) 


Main  Effect  of  Information  Type 


SOURCE  SUM-OF-SQUARES  DF 

HYPOTHESIS  740982.449  3 

ERROR  .137222E+08  28 


MEAN-SQUARE  F-RATIO  P 

246994.150  0.504  0.683 

490077.370 


Main  Effect  of  Weapon  Control  Mode 


SOURCE  SUM-OF-SQUARES  DF 

HYPOTHESIS  88171.879  1 

ERROR  1152728.047  28 


MEAN-SQUARE  F-RATIO  P 

88171.879  2.142  0.154 

41168.859 


Weapon  Control  Mode  bv  Information  Type  Interagticm 


SOURCE 

SUM-OF-SQUARES 

DF 

MEAN-SQUARE 

F-RATIO  P 

HYPOTHESIS 

4271.699 

3 

1423.900 

0.035  0.991 

ERROR 

1 152728.047 

28 

41168.859 

Main  Effect 

of  Blocks 

SOURCE 

SUM-OF-SQUARES 

DF 

MEAN-SQUARE 

F-RATIO  P 

HYPOTHESIS 

293679.543 

3 

97893.181 

8.743  0.000 

ERROR 

940500.953 

84 

11196.440 

Block  bv  Information  Type  Interaction 


SOURCE 

SUM-OF-SQUARES 

DF 

MEAN-SQUARE 

F-RATIO 

P 

HYPOTHESIS 

180928.879 

9 

20103.209 

1.796 

0.081 

ERROR 

940500.953 

84 

11196.440 

87 


SOURCE 

SUM-OF-SQUARES 

DF 

MEAN-SQUARE 

F-RATIO 

P 

HYPOTHESIS 

45197.543 

3 

15065.848 

1.267 

0.291 

ERROR 

999011.516 

84 

11892.994 

Weapon  Control  Mode  bv  Block  bv  Information  Tvne 
Interaction 


SOURCE 

SUM-OF-SQUARES 

DF 

MEAN-SQUARE 

F-RATIO  P 

HYPOTHESIS 

134516.816 

9 

14946.313 

1.257  0.273 

ERROR 

999011.516 

84 

11892.994 

Percent  C ha ag£_,( Arcurac t) 


SOURCE 

SUM-OF-SQUARES 

DF 

MEAN-SQUARE 

F-RATIO 

P 

INFOTYPE 

7.799 

3 

2.600 

0.192 

0.901 

ERROR 

379.940 

28 

13.569 

Percent  Change  (Response  Time) 


SOURCE 

SUM-OF-SQUARES 

DF 

MEAN-SQUARE 

F-RATIO  P 

HYPOTHESIS 

0.199 

3 

0.066 

5.897  0.003 

ERROR 

0.315 

28 

0.011 

APPENDIX  D 
EFFECTS  OF  PRACTICE 


A  second  analysis  looked  at  percent  change,  but  each  session 
was  broken  down  into  its  two  blocks  (not  tight  and  free  but  first  and 
second  in  temporal  sequence).  Table  9  shows  the  results  of  analyzing 
the  percent  change  in  this  fashion.  This  table  shows  the  percent 
change  both  within  the  Stimulus  Only  and  Priming  sessions  as  well  as 
the  percent  change  between  sessions. 


Table  9.  Percent  Change  in  Response  Times  within  and  between 
Stimulus  Only  and  Priming  Sessions. 


Percent  Change  in 

Response  Time 

Status 

-11.65 

-19.58 

-12.79 

Recommendation 

-5.17 

-22.04 

+1.67 

Status&Recommend.  -11.16 

-17.53 

-10.56 

The  percent 

change  within  the 

sessions  indicates  the 

proportion 

of  the  change  in  response  times  associated  with  practice  during  that 
session.  The  percent  change  between  the  two  sessions  indicates  the 
proportion  of  response  time  decrease  associated  with  the  decision 
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aid.  By  subtracting  the  average  of  the  within  session  changes  from 
the  between  session  change,  the  resulting  value  will  be  that  decrease 
in  response  time  due  to  the  decision  aid  for  that  particular 
information  type.  The  results  of  this  procedure  are: 

Status  =  -7.36  % 

Recommendation  =  -18.54  % 

Status  &  Recommendation  =  -6.67  % 

Figure  5  shows  that  the  slope  of  the  line  between  the  sessions 

is  steeper  than  the  slope  of  the  line  within  the  sessions.  Thus  the 
improvement  identified  in  the  previous  percent  change  analysis  is 
not  merely  the  result  of  practice,  but  also  a  function  of  the  assistance 
of  the  decision  aid. 


Response  Time  (msec) 


1200 


Session 


Figure  5.  Detailed  Percent  Change  within  and  between  Stimulus 
Only  and  Priming  Sessions  for  each  Information  Type. 
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